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SUMMARY 



•PROBLEM ^ t , 

Prototype computer-base^, multimedia, "individualized training / 
systems have demonstrated that the concept of computer-assisted tnstruc- 
tion/computer-managed instruction (CAI/CMI) is valuable and 1s directly 
applicable to an operational Air Force training environment. Sub- 
stantial savings in training times and more efficient utilization or 
resources have been demonstrated. However, the prototype- systems In- 
corporated research capabilities which were not essential to effective 
routine support of the majority of Air Force graining. The purposes 
of this study were (a) to identify those CAI/CMI functions with the 
greatest potential payoff for Air Force traini|g and (b) to develop a / 
functional specification for a low-ctfst compute-based system. 

APPROACH ' . 

•The applicability of and requirements for CAt and CMI In various 
Air Force training environments were determined by surveying potential 
user personnel. Computer system architectures were surveyed to . 
determine candidate systems to meet the functional requirements of .the 
low-cost CAI/CHI^system. Alternative terminal hardware devices 
communication systems were surveyed, in orde^r to recommend an economical 
set of input/output devices and communications interface to meet the 
functional requirements of the system. Variousxomputer programming 
lanquaqes were analyzed to identify a languag^wnich would-be cost 
effective for CAIVCMI programming. The CAI/CMI applications software 
area was surveyed, to determine the extent to which existing software 
might meet Air Force training needs. ' 

The initial assumptions uSed'to set the scope of the study Vere as 
follows: the computer, terminals, communications, programming language, 
and applications software must be capable of • . , ^ 

• 1. Supporting 500 students ^peV shift,; in 1500 hours of* Instruc-r. 
' Hon across five courses, with 1 50r instructional hours of CAI 
and with five CMI transactions, per student pec shift* 

I. Providing a proven student progress management- approach? 

'\ " 3v 'Supporting the oroductiQn and evaluation of instructional 

». 'materials for on-line and off-line user. • ' ' , . 

, vV ; 4! , Pwv1dlV« set of sl^^ard Reports for v ^°»?^|| v 0 ^ tr 
1 i - Force instructional personnel^ with' flexible capability to . 

retrieve and analyze graining performance d&ta for special ^ 

purpose- reports. • » 

. .... y • 



RESULTS 

Air Force instructors and supervisory personnel identified a broad 
range of functional capabilities' in the CAI and CMI area's with potential 
payoff for training. A total of 167 functions from seven major CAI /CMI 
functional areas are recommended for inclusion in the Air Force low- 
cost system. . '* ... 

A survey of central processing units identified a minicomputer con- 
figuration as "the principal candidate for meeting the functional require 
ments^of a dedicated local system." The concept of stand-alone devices 
was considered, briefly. It became" obvious th~at the myriad of combi- 
nations could not be dealt with within the -scope, and* time frame 67 the 
study. . ' 

A survey of terminals and communications resulted in recommenda- 
tions for a candidate low-cost student/admini strati ve terminal^ a low 
resolution graphics terminal with color CAI capability, ^ hi gtf ^ ' 
resolution graphics terminal for CA4» and a management terminal reader/ 
printer configuration ; • 

The survey of applications software indicated that all of the , 
recommended functions have been implemented on one or more existing 
computer-based instruction systems, demonstrating the technical 
feasibility 'of implementing the recommended functions. At the present- 
time, various computer-based systems present more* ®r less of the 
•recommended f uneti ons . Some of these ex vs ting systems, however, are 
'impleraentea^ on large centralized mainframes capable of- supporting " 
satellite operations, and incorporate sophisticated research and 
'development capabilities that are not essential to routine support of 
the -majority of Air Force training* 
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• * 1.0 INTRODUCTION « 

*• 1 " ***** 

Prototype computer-based individualized training systems have 
been developed and implemented at a number of Department of Defense 
(DOD) and civftten installations. One such training system is the 
Advanced Instructional System (AIS) at Lowry AFB. The results of a 
#ajnety of research and development (R&D) projects indicate that 

1.' Application of computer-based instruction (CBI) resulted in 
v substantial sauings in training time and more efficient 
utilization of resources. 
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2. 



* » 
An operational training system does not need sophisticated 
research capabilities which x although incorporated into pro- 
totype systems and required for research, are not essential 
to effective suppdrt of the majority of resident technical 
tra i ni ng operati onal .requi rements . 

3.' Full and economical exploitation of the demonstrated oper- 
ational capabilities of the prototype systems in routine 
support of resident training requires additional application 
* and system refinement. * ' * . , ; ' t 

Innovations in computer technology have drastically reduced the 
prices of computer components and terminals. Consequently, "the goals of 
computer-based training should te reexamined in the context -of 1979. 
state-of-the-art. computer technology. 

• • ■ • * . • 

The present study establishes the requirements for a purely oper- 
ational system and integrates those requirements into a system .design 
based on the best of current computer technology. The principal 
objective was to design a cost-effective system configuration which 
would efficiently support the CBI functions identified as having ; 
potential payoff from an operational viewpoint,* without including nice 
to hav,e ? ' or "rich" capabilities in support of R&D functions. Th v e 
products of this study are as follows: 

1 An overall system specification, delineating a set of 

functional requi rements, for a low cost computer-assisted in- 
* struct ion/cornputer-managea'* instruction (CAI/CHI) system 
(Appendix H), , - % ■ ; • .» . * • 

2. This Technical Report, presenting the assumptions, data 

collection methods, analyses, and results related tq.CAI and 
' mi system capabili ties,- selected system -architectures, 
communication systems, terminal hardware, and programming * 
languages and software. 'This report includes the configuration 
. " ''oran example low-cost CAl/CHl. system with cowmuni cations 

equipment, terminal, hardware, applications programs software,*. 
-. and a languag.«which will -satisfy tfte operati onal* requirements 
within •established constraints and assumptions. 
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2.0 STUDY PLAN ' • 

The research plan called for accomplishment of the following major 



1. Determine Functional Requirements 6f Users - survey Air Force 
* users, analyze survey data v and determine functional jjequire- 

ments for a Tow-cost CAI/CMI system fom operational use in Air 
, .Force Resident technical trainirfg programs. 



2. 

J 



Survey Hardware, Software, and Systems^* survey available' hard- 
ware, spftware, and languages, and* identify candidates for 
central processor units (GPUs), conniptions, terminals, 
software, and languages. 

• . 3. Document and Design - write a fu^M^ipnal specification and con- 
figure an example of a state-of-tne^rt hardware/ . 
communications/ terminals/software system to meet the functional 

• requirements. „ \ , " ( 

Figure 1 illustrates the task flow for this research plan, the 
following paragraphs describe procedures for accomplishing the tasks. 

In order to provide a framework around which the design work of 
this study could proceed, some assumptions were made regarding the if** 
ter\ded uses and environment for the low-cost CpFl/CWl Sysfeem. These 
assumptions are as follows: * *' z 

• » 

1. System design will be' oriented toward* resident technical train- 
ing. * . 

2. . System design wi>l be oriented toward a dedicated Tbcal system,* 

rather than a large centralized mainframe-oriented system or 
stand-alone mini system's . * A modular approach^ to expansion will 
be " incorporated, to facilitate tradeoffs *such as fewer on-line, 
students in exchange for heavier CAI usage. The effects of 

• increased loads on memory and mass storage requirement^ will 
* " • be considered* % t v 

3-. System design will be targeted toward providing operational 
■ CAT /CHI support for apug^ximafeTy 500 students per shift. \ It 
-is also assumed that:* the 500 students* are distributed across 
.five different courses; thre five courses account* for 1, £00 V 
, hours of instruction; % each student averages five.CMI trans- \ 
actions per sh.ift; pot more than W of the students are using 
CAI at any one. time; not more than^nine administrators (in- \ 
strtictors or^supervisory personnel) are on-line simultaneously; 
and batch processing is accomplished off -shift (i.e., during 
periods of low or no student* CAI /CM I lo/id). 
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*'•• X- 3.0 FUNCTIONAL RJEQU I 4JSER? > l% * 

VfhV initial step toward* designing a tow- cost CAl/CMt system was 
to determine those cornputer-provided ..f unctions whish* would be- of value 
in Air Torce train i^.gV To this eno\ the fuTlowirtg tasks Vere accom-, 
plishetf: 



V. "1 . Preparation; of a 'if sting of CAI and.CMI .functiops- which could 
' \ , ; 'b^.RroviaW by a rompiter- based tr^nijig system. 

Pitoaration ofa'ratirrg scale with whjch Air >Fdrce. users could- 
estate payoff 'potentials for various CAI/CHI functions.. 

- . •« . -\ 3 J Identi f feat ton of ;a sui ^abl e sample* of. potential Air Forge 

v; * users of"a low-cost CAI/CMI system. ' * * • - 

< • \ * ' .; V 

^4/ Wl^ction'of data from the sample of potential users. . j 

% \ ' ' , '•- ' "• • - : 

5, Analyses of the data. - , ' u * 

J6.- Determination of the CAI and GMI funqtians whictrshould.be pro- 
vided by the" low- cost system. 

3.1 i JJSTIHG^OF. CP/'CMI JUNCTIONS ••• 

V\ comprehensive "listing of 184 potential CAI- and CMI functions was 
developed, describing those functions which could be incorporated into 
a CBI system. This^ listing was -intended "to include any technically 
feasible- functions. which could* be provided for Air Forte training by a 
computer based 'system, and* which were not considered to be R&D 
functions; the^outcpmje>of this effort was agisting- of functions in 
seven major areas:' Student Testing and ^valuation, Student Assignment, 
Student, Progress Management, Support for Guidance and Counseling, CAI, 
Production and Maintenance of CHI Data Base and CAfe Materials, and In- 
formation and Reports (see Appendix A). 

The Appendix A lis|ing of the 184 functions^includes fche numbering 
scheme that- is, , used throughout this .report in referring to individuals 
CAI/cy'l functions--- Roman numerals designate the seven major areas, 
followed by letters and Arabic numbers that designate subcategories and 
individual functions. 

3 .2 PREPARE RATING 'SCALE AND. SURVEY FORMS \ ; 

•The 184 CAI/chl functions were incorporated into survey forms, 
with a rating scale Intended for use- by Air Force training and manage- 
ment .„ personnel in, estimating the relative values that- the functions 
would, have for operational training. The rating scale was oriented 
toward the concept of "payoff potential." . The introduction to the ' 
survey form and the explanation of the rating scale are shown "in 
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Appendix B. The ^explandtJon points out ihat payoff potential for: Air .» ■: 
Force training can coq« from any of the fallowing: 

.. ^ R^ucei' training ttm - 

« a Reduced. administrative load on instructors. 

More efficient jse of ^ training resources. \ - 

o improved quality control aver ^students- and/or courseware. 

Improved manaqemej^dmini strati on of training. 

Improved courseware' development and -eval uation j^cesses, - 
resulting in better instructional materials and reduced develop- 
ment times. ... \ 
.„"'•'."..••. . ; • * * 

' The instructions for using the rating scale (Appendix B) asked 
participants to consider how much each" of the functions would be worth 
to Air Force training within the context of the participant s duty . 
assignment, and to assign ratings using the following scale: 
. • . : * ' • " ' • . 

1 » Very high payoff potential. 

2*- High payoff .potential. 

3 = ^derate payoff potential.. 

4 = Low payoff potential 

5 * No payofV potential..- ; 

• X- No opinion * this function's, effects would be outside my area. * 
and I cannot estimate its payoff potential. 

Because* of the large number of CAI/CMI functions ^eluded in the 
listincu two survey forms were prepared. One forjn- include* xm . 
Sons ?hat were of potential value 

technical training at the instructor level. The second form included 

Motions mt were of potential ^-^-{«S^-X5q^ 
supervisors--e..g,, course supervisors, branch chiefs, and training 

advisors. ' "\ " : • v - 

The instructor survey' form asked for ratings on the. following items 
(see Appendix A): 

Category I (Student Testing and Evaluation) , all items except 
item IB. " . . 

Category II (Student Assignment), all items except item IID. 
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. Category III- ^Student Progress Management)', all" items* 

•Category' IV (Support for SWdance and Counseling), all items. 

Category V (Computer Aided Ins tract ion)!.' all items, v ' , ' { 

• ., Category VI (Production and* Maintenance of CMI Data pas^ and CAI 
Materials) all items except item VIC. * *• 

. 'Category V PI (Information Retrieval and' Reports) » all items' listed 
. under VIlA, Reports for. Instructors. " •♦ ? 

« - • . 

The supervisor ^survey form asked for ratings o*i t-he following i tems 
(see Appendix A): ' • * ' ' 

• * • . . •• 

Category I (Student Testing and Evaluation), items I A, IB, IC, 
and ID. . 

« 

Category II (Student Assignment), items JIA, IIS, I IC, lib, HE, " 
and IIP. • * ' 

. Category III (Student Progress Management), items I II A, 1 1 IS, • 
IIIB3-5, and ^ IIC. * ? f 

Category IV (Support for Guidance and Counseling), items IVA a-nd 
JVB. ■ . ' . 

Category V (Computer Assisted Instruction), items VA, VB1-7, and 

VC. . • . v 

Category VI (Production and Maintenance of CMI Data Base and CAI 
Materials), Items VIA1-2, VIB, VIC, VID1-6, VIE, VIE1-6 and 
VIP. • S . 

Category VII (Information Retrieval -and Reports), all items. 
■ ' \ • . .• • • 

Quantities of the survey forms were reproduced for u,se in the. data • 
collection effort.. Participants in the survey marked their responses 
directly on their* copies of the forms, and Identified themselves by duty 
position. Names or other specific identification were not required. . 

3.3 SAMPLING .OF POTENTIAL USERS 

Although the design of the low-cost systenfwas to be, w oriented . 
principally toward the needs of resident technical, training, the needs 
of. other Air Force training programs were also to be .identified. This 
would permit documenting the extent to which the* low-cost system would 
mee't the needs of, for example, on-the-job training (OdT), navigator 
training, and field training detachments (FTDs). Therefore, samplings 
of potential -users from 8 resident technical training, and from 'a number 



<*f other. Air Force training programs were identified for .inclusion in 
the survey of user requirements. ^ / , * 

• 3.3. 1 Resident Technical Training Sample. *V ; ' 



v 



The sample of resfdent technical training personnel was • 
drawn from Air Training Command (ATC> Headquarters at Randolph AFB and 
from four A1r Force Technical Training Centers (TTCs)--Sheppard, 
Keesler, Chanute, and Lowry. 

During the'AIS development contract, courses at the TTCs were 
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^requirements: 

Shepparti'TTC • , , . , • . 

Air 'Cargo Specialist (3BRB60531) / 
Disbursement Accounting Specialist (3ABR67232) 
Faculty development (3AIR75100-4) ... 

Keesler TTC * * 

^ " Personnel Specialist (3ABR73230) 

. . Ground Radio Equipment Mechanic (3ABR30434) . 

Telecommunication System Control Specialist (3A3R39730) 
, Faculty Development (3AIR75100-4) 

Chanute TTC * \ 

Aerospace Ground Equipment Repairman (3&BR42335) ^ 
Life Support Specialist (3ABfc92230) ^t* 
Airframe Repair Specialist (3ABR42735) 



Lowry TTC _} 
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.Munitions Maintenance Specialist (3ABR46130) 
• Inventory Management Supervisor (3ABR64570-1 ) 
Faculty Development (3AIR75100-4) . 

• Personnel from several other ATC resident technical tracing pro 
grams ware included in the user survey because of their involvement 
with, and knowledge of, Air Force computer-based 'training; 

^eppard.. TTC" (users of PLATO) , * * 

' Heal th^re. Sciences 



' / Aircraft Pneudraul ics \( 3ABR32334) V . 
. . Ran? and Requir«nents\ Division | 
, PVATQ. development Gmup * 



Lowry TTC* (users of^AIS) \ p ^ - . • , 

* "• •* Inventory Management Sdecialfst (3iBR64^0) 
Materiel Fa?i1 1 i ti es Spetfaiist (IABR6453U 
*" Precision Measuring Equipment specialist (3ABR3243Q) ^ 
Weapons Mechanic (3ASR4&230).' 

•• •• • **r 

The Air, Force memoranda requesting the TTCs 1 participation «in the< 
survey of user needs requested "...cooperation in the Feasibility Study 
Survey of the following Individuals/agencies involved with conduct of 
the above courses: . . , * * 

I 1. The Deputy Commander for Training 
ir 2. The Training Advisor * , . 

3. The Technical Training Group Ct)mmarider(s)' • : , 
. • 4. The Chief Plans/Requirements Division 

5v The Chief, Branch Curriculum DeveTopraent Unit 
-Resident Course Representatives 

-Nonresident Course Representatives # . 
^Instructor and Course- Evaluation (I&M) Representatives » 

6. Course Supervisors of the above Courses 

7. Ten Instructors^ f rom each of the above Courses." . 4 . 



The participation of personnel from Technical Training (T"T) and 




. 3^3:2 -Other A'ir Force Training Sample .;, £ 

The following instructors and supervisors from training 
programs other than resident technical training were identified* for in- 
clusion in the survey of user requirements: * % 

T; TTD/01T instructors ^and supervisors f row Sheppard, Mather, and 
Davis Monthan .AFBs, 

2. Program managers and staff personnel *f rep .Air University. 

3. Staff personnel f rom HQ ATC (00),* Rudolph AFB (pilot, training) 



' * * * * 4, Instructors and. staff personnel from* Undergraduate* Navigator, 
; ? Training (UNT); Mather AFBr . '/ \ ' " • ; * , 
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} : $A' pATA* COLLECT 

The necessaiy survey data were* collected at- each' of the .sites- • . * . /'".•'»- 
v 'identified as part 0/ the user survey samplinc* Tlje preferred pre* . 

xedure, observed wherever the participating personnel could accommodate ^ 
the demands on their time; was as follows: .a . ... . . .„ 

1. A group of " Air Vo»:ce parti ei pants gathered f or a>30-rai nute • Vf 
' ' . . briefing which explained the* purposes of the survey and the 
■"*/•** . ' meanings of : GAIy 011, and other; special terms used in the . \ ' ;: t f 

' survey :for^is . • • • * ' , ; \"> ■• •'**. ' . * 

2. Survey forms were distributed to the participants.; r ' . ' * * 
3 f Participants' completed the survey forms. • • * 

' 4. The survey forms were 'collected. '. ' 

" 'lie total time required for" this procedure averaged approximately 
2 hours. Personnel who could not- stay for the entire session were^ y \ 
• permitted to take .their survey fdrms with them to be completed later. 
In these cases, a representative was designated to collect the forms. 
Some Lowr/TTC personnel were already familiar with v A IS computer-based . / 
training, and the 'briefing was emitted. "Using. these procedures, the ' 
rate of return for survey fe.rms was better than 96%. . * \ 

For some training programs, other than resident technical training, 
the survey foms were "not --deemed appropriate. In t^ese cases, the 
participants were- interviewed to determine their particular functional • 
1 retirements for a computer-based trainino system. , 

- 3.5 DATA 4WLYSI5 PROCE&JTFES - * . 

? functions extracted fro.- the *cc:«i?leted survey forms-were entered into 
• the'AIS computer. TheK for each Iter- in the listing of functions, 
' mean payoff r^ings and distributions of ratings by groupings of re- . 
spondents were derived. The grouping of princip-a Tenter est for the 
* ; desi gn phase uf this study was "al l AT$ resist teeJmical training. 

.respondents. 1 * -Groupings* oSf ATC respondents involved wttH trapping pro- , 
grams ether titan resident technical training were of interest in< • , 
determining the .extent, to which the low-cost system would be able fci 
&eet the needs, of other types of Air Force training. 

The various groups included in the survey differed in si 2S, 
ranging fmm 1 to 23 participants per* group.. It was assumed that all* 
gjaJups were of equal, importance and should contribute equally to tne 
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to thp oveWl mean ratings^ 

The user ratings of the CAI/GMI functions arid the inierrimpktti: 1 ' 




each function ,that ft be either retained as a function to be provide^ 
by 'trie low- cost system or eliminated froa further consideration". 

3.1-.1 Results From Resident Technical Training. 

•* • ' '' , . ; * • ..••r.jp,-'-. 

; A total of 268 personnel involved in resident techntiarfc > 

training completed forms for the survey. of user functional requiretrSts. 
The jistnbutlon of participants is,shown in Table J.".' 

The results frcn the survey of ATC resident technical 'training 
progrUis an» shown in Appendixes C and D. Af5>en4ix C >ists the 
f actions in the- order in which they appeared on the survey T orris and 
indicates the mean payoff ratings aW the, system -Inlets "{files needed 
and estimated CPU usage), for each of-the functions. Appendix D lists 
the functions in order of average estimated payoff, and includes the 
distribution of ratings given t6 the functions by the survey oarticf- 
panus . . » 

The user survey results can be suRwarized as follows: 

< * * ' * ' • " \ 

V. f4ead. ratings of the 134 functions range Yrotn the highest- 
ratjed item (Flag Test Itam Alternatives Missed by More Than 
70% of Students) wi^i & r.1ean rating of Z'.07; to- the lowest- 
rated item (Students Select Desired Alternate Module As sign- 
- ' .ments) with a mean. rating of 3.71. ' - 



2. Of "the. 181 listed "functions, 61 received "average ratings ' \ 
better than 2.50, and 115 received average "ratings less than 
2.130 but better less 3. CO. OnTy S of the functions rated 
worse than 3.00 (moderate payoff). t • 

3. . Eighty;- two .of the functions were r&ted 1 (very high payoff) or 

2 (high payoff) by 50% or more of the 268 participants. 

4. Even the lowest-rated functions (Students v Select Alternatives, 
and Heuristics flules For Assignment Selection) were rated hjgh 
6r very High payoff by more than 7% of the participants. 
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; TABLE"! . RESIDENT TECHNICAL TRAINING PROGRAMS. AND 
« . NUMBERS OF PEOPLE SURVEYED - 

LOCATION/PROGR AM T _> • ' ' .* INSTRUCTORS SUPHWIS ORS 'TOTALS 
.'SHEPPARD TTC . ; • » . V . *. . 

Air Cargo Specialist . . 9 . . 8 • « ,■ % 

Faculty ^Development 11 , *. : '10. 

4 * ' Healthcare .Sciences * % .9 .* 10* , 

':>,<•;•;•" • ... ' • • Total • « • 

..KEESLER TfC. ' * % 

• •> . • ' .... • .*« : • , - 

Ground Radio Repairman .7 7 v 

Personnel Specialist 9 6 . > 

« r Faculty Development - 4 

Telecommunication Specialist 5 4^ 

7 Total . 42 

V CHANUTE TTC ' \ . : . 

v , \ 

AGE Repairman 8 15 

* Vehicle Repairman 15 3 

Life. Support Specialist 4 - 7 

< Aircraft Pneudraulics 10 3 

Airframe 'Repair Specialist 5 4 

• Plans and Requirements Di v. ; * , •• J 

, • PLATO Development Group * - 4 

\ ' Total . . • ■ 84 

LOWRY TTC $ . 

Inventory Management , 9 1 

Materiel Facilities • 6 
Pd||i sion- Measurement Equi p . 
inHntory Mgt. Specialist 

• . Munitions Ma'int. Specialist , - i % 

Weapons. Mechanic* .. , 5 . * / c 

^ Faculty Devel opment ; , 3 , & -V 

Total 

* RANDOLPH. AFB (HQ ATC) '* 

Technical, Training (TT) ' , - ' }\ 

. ; Plans (XP) % " * 3 iA 

Total ' .I 4 

ToVaLS . f38 1 30 ^ 268 
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• - 5. T*e highest-rated furKttons (Flag Alternatives Missed By More , 
Than 70%), was rated no payoff by only one of the, participants. 

* * A ' ' * . 

6. The, numbers of X (don't Rnow) responses, ranged from, 4 (for On 
Line Test Capabilities)- to 55 (for CAI Authoring. tditor - 
•Automatically Structures Sequencing Of Frames), •> 

Mean ratings anc| correlations, between ratings from th^ various 
resident training organizations* involved %a re shown fn Table^2. The 
overall mean ratings range from 2.91 (slightly better thap moderate 
payoff^verage for the 184 functions') at Lowry u> to 2.48 (toidway . 
between moderate^ahd high payoff.) at Keesler. The correlations . §re a-11 * 
positive and range from 0.30. 'for l^owry/ Randolph up to 0.78 for Sheppard/ 
.Keesler ,'" There is moderate 'agreement between organi zations in 'their 
ratings of the functions and quite close correspondence between organ- 
izations in their overall me^ah ratings. In other words, the organ-* 
izations showed moderate agreement as to which functions were most, and 
least, important, and agreed rather closely as. to the potential payoff 
of CAI/CMI for their particular training operations. „ * ■ 

TABLE 2. MEAN RATINGS F-ROM RESIDENT TRAINING ORGANIZATIONS 
AND CORRELATIONS BETWEEN "ORGAN IZATIONS * 



LOCATION **MEA|U CORRELATIONS 

RATlW KEESLER CHANUTE . LOWRY RANDOLPH 



SHEPPARD 2.50 .78 . .57 .41 , .39 

KEESLER 2.48 .43 *' .36- .44 

CHANUTE 2.62 * • . . . .63 ,33 

LOWRY / 2,91 • 1 • • % .30 

RANDOLPH 2.78' 

Mean ratings. from personnel with and without CBI experience at 
LQwr^ and at Onanute are shown .in Table 3. These comparisons are of 
interest in determining if 'experience with CBI results in afny overall 
change in* how the payoffs of CAI/CMI functions are viewed. -At Lowry 
TTC, the average rati of from personnel in AIS courses was 2.70, some- , 
what better than the 3i13 average for personnel in the non-PLATO. 
courses. At Chanute, the overall mean rating from personnel fn PLATO 
courses was 2.37 , somewhat better than thw overall' mean of 2."S8 from' . 
personnel in the non-PLATO courses. It appears that experience with €BI 
did not markedly affect the mean ratings of ^CAI/CMI functions. / 



T£BLE 3. MEAN RATINGS, FROM PERSONNEL ASSOCIATED WITH 
. .. ' COMPUTER-BASED AND NON-COMPUTER-BASED "COURSES . 



WRY JTC 
AIS COURSES 



s MEAU J NON-AIS COURSES 



.MIAN 



ventary Management 

terfel Facilities 

'.'"«' i • * 

ecis-.«Meas. Equip. , 
t ' » . ' ■ 
' apons Mechanic 

Overall Means „ 



2.88 1 
2.51 , 
2.89 
2.52 

2.70 



inventory Mgt. "Supvr. 
/Muni fri otts" Ma i ntenarrce 
Faculty Development- 



f 3.4$ - 
3.25 
2.69 - 

3.13 * 



IANUTE TTC 








PLATO COURSES 


MEAN 

> 


NON-PLATO COURSES •*• 

» * 


MEAN" 


ihicle Repairman 


,3.06 


AGE Repairman 


2.45, 


teudraulics 


2.63 


Life Support 


2.5# 


• * 

JITO Devel. Group * 


1.41 


Airframe Repair 


-2.70 


Overall Means 


2.37 


i 


, 2.58- 



The last item on the survey 'forms asked participants to list any 
dditional functions that should be provided by a computer-based train-, 
ng system. There were 57 participants who responded to tips 
uestion. Most of the responses added more detail *te already-listed 
unctions, or commented (positively or negatively) on the survey and on 
81* The substantive suggestions for additional functions, and the sub 
equent actions taken, are as follows: 
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System* to ad jus t\the honor graduate score to keep within the 
10% ATC. requirement - Action i Xo be included in the low-cost 
system specification. * 



System to provide a data base to handle credit in the 
Community College of the Air Force JCCAF) for courses 
completed - Action: not to be included in low-cos,t system, 
because of *potenti al i nterfa.ce problems, and because thje 
low-tcost system is to be a local system. 

System to provide jHaTHup conversational capability between 
terminals, for student-instructor interactions - Action: not 
to be included in low-cost system, because of likelihood of 



misuse ah'd because the initial low-co*st system is oriented*, 
toward local resident technical training use, rajther thajr ' - 
• terminals /distributed at remote- Locations. ■* j> 1 * 

v ■ »*.-', ' 

\ System to assign instructors automajti call y, within the Course, 
based on qualifications, availability, etc. - Action: not 1 t'o^ 
be included because, of unfavorable, institutional change 
I aspects and overlap with, other- Air ^orce personnel management 

• systems. * ' .* • . 

.... . . • . / t * 

• • 5. System "to assign graduating students to duty stations, based 

, on student proficlencies/deficiencies and on duty station 

requirements - Action: /not -.to be included, because of unfavor- 
able institutional change aspects and overlap with other * 
personnel management systems. ' ' 
. . • •' 

"Examination of the survey results leads to the conclusion that all 
f the CAI/CMI .functions, are of some potential value to resident 
echnical* training. - Furthermore, estimates of payoff potential range 
long a continuum, without clear separation points, from low payoff up 
0 high payoff. The moderate correlations between organizations suggest 
i verse training needs — i.e., the subset of functions with the great- 
st payoff potential for one AJC organization is not the subset with 
.reatest potential for another organization. The system impact 
•stimates shown in Appendix C indicate that file requirements and CPU 
imes are determined principally by inclusion or omission of major 
iroups of functions, not by inclusion or. omission of a few functions 
rOm within a major groups Therefore, even though specific functional 
•equirements differ from program to program, it may be most practical 
:o provide a system which has all desired capabilities and allow the - 
>rograms us-e those functions they want and ignore othe/s. 

.As* the final step in the analysis* of functional requirements for 
VTC resident technical training, AIS experience and .lessons learned 
rere applied in determining the final list of CAJ/CMI functions tb 
inter into the system analysis for the Tow-cost system. The functions 
;hat were dropped from further consideration, and the rationales for 
jmitting these functions from the low-cost system, are listed in Table 

The retained functions enter into the^system analysis and cesign 
:>hase of this study. - 



* * * ' TABLE 4. -FUNCTIONS OMITTEi FROM THE LOW-COST 

' \ SYSTEH SPECIFICATION AND .RATIONALES FOR OMISSIONS * 

FUNCTION (Complete Statements are fn Appendix A) /RAT I ON ALE FOR OMISSION' 

1 V IA2 STUDENT TESTING AND EVALUATION > CORRECT SCORE ' FOR GUESSING/ . 

Rated tow in survey of Air F6rce needs, npt viewed .as - • 
* educationally important by ATC personnel . - - 

2 I A3- STUDENT TESTING -MfO "EV^LUATION-ICIGHT QUESTIONS •DIFFERENTIALLY/ 
: . 44ot, practical »>14\ Air Force courses to determine the wei ght- 

- ings. Requires "subjective judgements f rofli -subject matter , 
experts, *or requires a large amount of performance data • ^ 
collected within the course and from the fieltf, with time-' 
consuming data analyses and interpretations to relate 
• ' s responses with field 'performance. "The demand on, course. 

personnel to, accomplish tjiese-tasks is not worthwhile, in 
terms of educational benefits." 

3" IA6 "STUDENT TESTING AND EVALUATI0N~PASS SPECIFIC QUESTIONS/ 

, Rated low by ATC personnel, viewed as low payoff. Determin- 
ing critical "must answer" quest ions- requires" subjective 
« judgements by course personnel , supplemented with data 

collection and analyses to validate the decisions. The ed- 
ucational returns will hot justify the. demands on ATC course 
personnel.' . } , •• J ' . * w 

4 IB4 STUDENT TESTING AND EVALUATION-ON-LINE CONSTRUCTED RESPONSES/ 
The costs of software development. :for thi s capabi 1 ity , and" 
the demands made on authors, outweigh- the educational gams 
that tan be expected in ATC resident technical training. 
Authors are- required to specify, in great detail, allowable 
alternate' spell ings i alternate oriierings of Words, etc. 
f Only afteV^carefuT Observations of student responses* can an 
' author determine that the correct, answers are all treated as 
correct and that al 1 the incorrect answers are scored, as 
incorrect. £• ' 

* * ' * - . * 

5 4 IB6 STUDENT TESTIT^AND EVALUATION ON-LINE, ADAPTIVE Cfi* C.OURSE , 

PERFORMANCF/ * . '■ ,\ * * mIV' 

. Too difficult for authors- to define the adaptation to be 
usVd. Requires data, and'analyses, to relate- student 
performance on other lessons in ^the course to 'what should 
be don* at this step in this test?" The necessary analyses 
^ and interpretations are, practically, not cost effectrve 
^ in ATC resident technical training 

* 6 UBS STUDENT ASS I GNfENT-ORDER SELECTED BY INSTRUCTOR/ ' 

This capability, if provided, would impose on ^instructors 
the task of determining the sequence, in which each student 
should take the course lessons. It is more practical to 



. • * TABLE # 4. . (Continued) 

- t have the system make assignments based on criteria 'deter- 
• mined by*lie,&rancn* t with instructors overriding for * 
exceptional cases. * j£> «. 

7. , IIC1 STUDENT .ASSIGNMENT- INSTRUCTOR ASSIGNS ALTERNATES/ < " 

As -for the preceding item, the instructor is freed of routine 
if the system makes the assignments to alternates, witn" the * 

instructor overriding for exceptional -oases s ' « •• 

' ■ • - *♦ • • • ■••.*' ■ . '*-.>'••.... . *. 

3; IJLC2 hVmm ASSIGNMENT-STUDENT SELECTS DESfRD? ALTERNATES/ ..; 
" , Rated very low by ATC— instructional payoff is believed to be" 
quite smalK Omitting this function still leaves the in- - 
:f structors free td -override, so that a student who can con- 
vince an instructor to intervene may take any. desired 
alternate. s * « * • 



• 9. II-IB5 STUDENT PROGRESS f1ANA6EM|||T-GRADUATES: SCHEDULE OUT- 
PROCESSING/ • » , 
» * The system will flag those students who are to graduate 

' during some designated time period. However, the problems 
u of interfacing with all of the organizations involved in out- 

processing students is outside the scope of the low-cost 
system effort. 4 

. .. 10: IIIC6 ST.UDENT PROGRESS MANAGEMENT-MANAGEMENT TARGEJS RELATIVE TO 

« PLAN OF INSTRUCTION LENGTH/ . ,. : 

Average course length, is more immediately related to 
~ curriculum changes than is Plan of Instruction length." 

Therefore, management functions which are relative to ' 
average course length will reflect shortened or lengthened s 
.. curricula sooner than if relative to Plan of Instructfon- - ; 

* !. length. 

11. VB4 COMPUTER- AIDED INSTRUCTION (CAI) -TOUCH PANEL OR LIGHT PEN/ ; 
> Rated low by ATC personnel. Seldom needed for student use. 

.* / ' Principal value is in preparation of graphics j displays for , 

1 . CAI lessons, r This will be accomplished at a graphics 

terminal, in the low-cost ^system, rather, than at a • student 
. . Ihteracti ve . terminal \ 

' " 12. VII61 INFORMATION RETRIEVAL AND REPORTS- PRESER VICE- EQUCATIOJJ, VOC. 

» TRNG., TO MAKE FORECASTS OF FIELD fffRFQRMANCE/ I . ' 

Dfff J«iri£.to -'derive jiredicttons based on fcrsr-service V: - 
» vocational education data. Good predictions would require, 

for each course, a sufficient data base from which to derive 
reasonably accurate regression equations or heuristic rules, 
V" and the skill and time to perform and interpret "the Analyses. 

'..-.' • i . • • ., *- ;' < . 

; ' y , . . ■ ' •- ■ " - , V- 

; . . 22 rt ..." v 
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TAIJLE 4. (Concluded) ' . > ; 

13. v VIIG2 INFORMATION' RETRIEVAL AND ^Rfi^RT S-PRESE^VICE OCCUPATION AN& r 

■ JOB EXPERIENCE TO MA*E FORECASTS%i)F FlELO PERFORMANCE/ *' 
• 'Rationale lor omission, same, as °For VHSl above. 

14. VllSa INFORMATION RETRIEVAL AND *REPORTS-Afv PERSONNEL WECCi^DAT^. 
. (ASVAfr.ETC.) TO. MAKE FORECASTS OF, FIELD PERFORMANCE/ 

• * * Rationale for omission, same is for VI Hah above. 

15. VIIG5* INFORMATION RETRIEVAL AND REPORTS-SPEC I ALTY-REtATtO 7^ 

. • DEFICIENCIES TO-MAKE FORECASTS, OF ^IELD ^RF0RMNCE& 
, J Rationale for offlfssion, same asf for VIIG1 above it ;. r 

16. VITG6 INFORMATION RETRIEVAL AND REPORTS-PREDICT STUDENTS' 

ADAPTABILITY, PERFORMANCE, ANO PROGRESSION IN OPERATIONAL 

FIELD "ASSIGNMENTS/ * *- 
Ratlonale for omissionj sane as for VIIG1 above. 

NOTE: Of* the VUG set of functions (Performance Forecast Reports to' the 
Field), VIIG4 wil'l be provided by- the low-cost system Reports 
to the field based on .in-service training performance data. ' 
- ■ - . * ,* * ;• * . • ,• 

17. VIIH1 SPECIAL -REPORTS-BASIC DATA RETRIEVAL. BKGD/ 

Dropped in fa'vor of the higher-rated Interactive 
.Retrieval With. Statistics (VITH3). - " ; 

18. VIIH2 SPECIAL REPORTS-fNTERACTII/E RETRIEVAL/ . 

Dropped in favor of the higher-rated Interactive >, 
' Retrieval With Statistics (V1IH3). N " \ 
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[3,6.2 Results from Other Air Force Training Programs 

*' • 

. In addition to surveying ATC resident technical training 
requirements, other Air Force training programs. were also included. 
The .additional program^, and numbers of 'participants, are listed in 
Table 5. As described earlier (Se6t ion 3.4.1} and as indicated below, 
data were . collected using survey forms and by interviews in which the 
Survey form served as a guide. * „ 

TABLE 5. ADDITIONAL PROGRAMS IJI6LUDED IN SURVEY, AND 
NUMBERS t)F PARTICIPANTS 



"FIELD TRAINING DETACHMENTS & ON- 
THE-JOB TRAINING--SHEPPARD,* 
MATHER, & DAVIS MONTHAN AFBs 



INSTRUCTORS SUPERVISORS 



Survey Forms 23* , 9 

^Interviews , ^ " , $ 

AIR UNIVERSITY » 

Survey Forms ^ - * , 11 

EXTENSION COURSE* INSTITUTE - * ; 

. Interviews ** , . 8 

HQ ATC (DO), RANDOLPH AFB , 
6Pi lot .Training)* 

Survey Forms " * ' , - 1 



MAJNE| AFB (Undergraduate * 
Navigator Training), * 



Survey , Forms .2 5 

Interviews * 6 11- 



f * - TOTALS. • 33 50. 

* Explicit definition of the computer-based functions required by . 
Flight Training |s given in Data "Automation Requirement DAR A78-2, 
Time Related Instructional Management for Pilot Training, Air 
Training Command, 29 June 1978, 



Results from th# survey forms are tabulated in Appendix E. The, 
mean ratings, and the correlations between ratings from the various pro- 
grams-are shown vin. Table 6.. Reside* technical training is also 
Included in Table 4 6 for purposes of comparison. 



PROGRAM* 



TABLE 6, 



OVERALL 
MEAN 
RATINGS 



MEAN RATINGS AfiD CORRELATIONS FOR VARIOUS 
AIR FORCE TRAINING PROGRAMS ' 



CORRELATIONS 



UNT 



PT 



AU 



RES 



OJT/FTD 
UNT 
PT 
AU 

RES- 



2.92 
2.68 
3.09 
3.31 
2.62 



.25 



.04 

.73 



.34 
.08 
.16 



.37, 
,13 
-.08 
.5 V 



> On-The-Job Training/Field Training Detachments (OJT/FTD), Unier- 
graduate Navigator Training (UNT), Pilot Training (PT), and tar . 
University (AU). Resident Technical Training (RES) is included for 
comparison. * , . 



The overall mean of the ratings from resident technical training 
is 2.62 — between Hi § h (2.0) and Moderate (3(01 Myoff. Jh? mean 
rating 'from UNT is down only slightly (2.68), fo lowed by JMJ/HD . 
(2 92), PT (3.09), and AU (3.31 )♦ w These other Air Force traming^ro- 
g^msldo see payoff potential in the CAI/CMI functions, but not a* much 
# potential as in resident .technical training. The correlations indtcate 
Some comnonality among -the needs of OJT/FTD, AUVand resident P^^} 
training (correlations of .34, -37, and .51 ). There is a so consider- 
able commonality between navigator trainfng and pilot training 
(correlation of .73). ^. v "s 

The notes from the interviewsrwere analyzed, and the results are 
shown in Appendix F, Because the survey form served as a guidejor the 
interviews, the responses fall into the seven categories used on the 
survey form but do not correspond exactly, to the wordings used in the 

. sur'vej Each function listed in Ap^j)^ «w.^^a z ^ « ei ^ r i: 

(a) already provided by the functions listed on the survey form, (b) 
already provided by the survey form functions but requiring-minor 
modification to Mtept from resident technical training (for example, _ 
referencing test^eW to Career Development* Course volumes instead pf 
President training lessons, o/(cE not provided by the purvey form 

• functions. These* categories are also indicated in Appendix F. 
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4.0 HARDWARE SURVEYS AND ANALYSES * 

_ To provide a framework for. configuring, a low-cost system, the 
following training scenario and design .goals were established: 

1. Approximately 500 students per shift* enrolled in five courses. 

Not more than 10% of the students usik Iht at any one time.* » 

An average of five CMI interactions per student per shifty 

Not more than ten administrative personnel on-line at any 
one time. . . .\ . 

5. Productive response times as follows: * 

a. less than or equal, to 1 second average response' time for 
. simple, operations. 

b. less than or equal to 3 seconds average response time for' 
more complex operations.* 

The training scenario and design goals suggest c a probable price 
range iW9) of % 4500,000 -to $600,000 for mainframe, peripherals; 
terminals, and associated support hardware. . 

Surveys of hardware were restricted to .selected equipment items in 
the areas of fulj size and minicomputer mainframes and peripherals, data 
processing equipment, terminals, and related coitiauni cation systems. 
Equipment was evaluated as to its potential fwf» application- in the .low- 
cost system. The survey methods included reviews of literature, trips 
to remote. sites to consider classes of equipment or specific hardware, * 
and visits to trade shows and seminars. The DataPro Research Corp. ' 
(1978) and other surveys furnished preliminary reviews of equipment 
manufacturers' specifications on computer mainframes and peripherals, 
termi nal s , commuhi ca ti ons systems , and software . 

4 J CANDIDATE CPU ARCHITECTURES 

" The purpose of this portion of thfcr study was to analyze existing 
computing architectures and to identify one which would provide a oost 
effective approach to supporting ATC resident technical training pro- 
grams. To accomplish' me initial investigation tasks, current pub- x 
locations on existing computing architectures were examined. Then, 
considering functional capabilities and acquisition costs, a number of 
existing computer architectures were selected as an initial set of can- 
didate CPUs for more detailed investigation. Data on the efficiencies 
of each of the systems were analyzed to determine the b'est performing 
CPUs. Then, the criteria of software quality and reliability, hardware 
reliability, and effectiveness of maintenance service were considered 
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as additional evaluation factors . * • . 

Within the framework of the training scenario, design goals, and 
evaluation factors described- above, many large-scale and super* computers 
could fulfill the functional requirements. However, the CPU and input/ 
output (I/O) power they provide are not necessary in thts application, 
and tbe$osts of *suah machines" are above .ttie expected price range. Many 
smaller-scale computers are priced attractively, but dp not provide the 
necessary power. 

In evaluating candidate architectures,- millions of operations per 
second (MOPS) was used as the unit of measurement, rather thap millions 
of instructions >er second- (MIPS), becaus'e the* MIPS rating can be mis- 
leading. Machine .1 may have a higher MIPS rate than* machine 2, but it 
may take machine 1 more ^instructions to do a given amount of work. 
Machine 2 could actually have a MIPS rate that is less than machine 1 
and' yet accomplish an entity of work- In a shorter period olj time. 

A MOPS figure of 0,15 (150,000 operations per second) and an"I/0 
rate of 24.0 (24 million bits per second) were es tab! i shed- as minimums 
needed to meet the functional requirements for the low-cost system. 
These figures were arrived at* in the following manner. In the current 
AIS system, the hardware provides functional capabilities that are much 
in line with the training scenario and design goals described;-- On the 
current AIS there. are, normally, approximately 500 students, approxi- 
mately 50 on-line 'terminals (including CMI terminals) , and response 
times similar to those given previously. Therefore, the capabilities of 
the current AIS system are known-. The CDC 6400 (equivalent to the AIS 
Cyber 73) which was selected as the benchmark system, yielded MOPS «* 
0.17 and I/O rate - 24.0. For adequate performance in t|ie low-cost , 
system, a MOPS figure of 0.15 and an I/O, rate of 24.0 were determined 
to be' rainikims below which unacceptable degradation of performance would 
result. " n * / , , 

After evaluating the candidate CPUs, tne Digital • Equipment Corp: 
(DEC) VAX 11/780 was selected for purposes of developing a system design 
and cost analysis for the low-cost CAI/CMI system. Although this is not 
the on-ly CPU that will meet the functional requirements for the low*cost 
system, the VAX 11/780 serves &s the example CPU in subsequent sections 
of this study. : 

If the functional capabilities required of the low-cost system were 
reduced," some torr^spbna^hgtdecreaise in computing power tjjfd tyi CPtt 
cost) could be realized. For example, if it were desirecpto support the 
same students but with terminal response times of 10 secdps- instead of 
the faster times, assumed as goals for the low-cost system! then a less 
capable CPU could be-choseh. It should be kept in mind, however*; that 
the mainframe represents only about 15% of the cost of the system. Re- 
ducing the computational* capability of the system would reduce 'the cost, 
but the responsiveness of the systerii would also be reduced. For these ' 



reasons, it is more* cost effective to choose a system which will yield 
the higher level of productivity at a small increase in cost. The 
terminals represent about 50% of the cost of the system. Major cost 
reductions can only be achieved by eliminating major areas of 
capability (CAl or CMI, for example h thereby reducing the cost for 
terminals. • , ' 

4*2/ SURVEY OF INTERACTIVE TERfHNALS » . .;" t ...'"*' 

The capabilities required from interactive .terminals by students, 
authors, instructors', and other administrative personnel were analyzed. 
Administrators need alphanumeric (text) presentations, principally, and' 
have very little need for any additional "cafJabiTi ties. Preparation and 
presentation of CAI lessons, and on-line preparation of materials for * 
off-line presentation, also rely principally on alphanumeric text 
capability. However, .there, are some content areas and special presen- 
tations whieh will benefit from color and graphics capabilities.* Ih , 
general , CAT programs require upper and tower case characters, numbers, 
and 'punctuation from the American Standard Code for information Inter- 
change (ASCII) set, and a -next- key for student control over the pro- 
gram, programs for CMI have the same requirements, plus needs fpr 
additional function keys (-back- and at least $hree additional function 
or control keys). 

The attributes of some 300 alphanumeric display terminals and. 
graphics terminals were reviewed for similarities and for industry-w'ide 
standard features. The results (below) indicate constraints within 
which the low-cost system will have to fee designed if off-the-shelf 
terminals are to be used*. The relative merits of various terminals were 
evaluated, and candidate terminals ar^ recommended for oise by students, 
authors, and administrator in the low- cost CAI/CMI system. 
/. ■ * . • • 

4.2.1 Display Format * m 

•» ■ 

A format of SO characters per line is standard for most*- 
terminals. Alphanumeric terminals commonly provide 24 or 25 lines, and 
graphics terminals commonly provide 34 or 48 lines. Most terminals 
prgvide as standard either 64 different displayable characters (numbers, 
upper case alphabet, and limited punctuation), or the standard 96 charr 
acter ASCII set (numbers, upper and lower case alphabet, , and e^tra 
symbols). "The 96 characters are considered necessary for the text 
requirements of the low-cost CAI/CMI system. ' 

Symbols are formed within a dot matrix on all the terminals, t 
Characters using higher resolution formats are generally more readable, 
especially for such characters as M m tt and "w." Lower-case letters are 
easier to read when the descenders for the characters M g, n "j," M p," 

"q," and "y" extend below the matrix used with capitals and numbers** 

. - • . «- 

The low-cost graphics terminals reviewed have a picture element 



(frixel) of resolution which is some fraction' of a one-character matrix. 
The range of resolution standardly available ranges from the lowest, 
160 pixels horizontally by 192 pixels vertically, up to the highest 
resolution', 640 horizontally by 476 vertically. 

Many programs used on the low-cost CAI/CMI system'wotild benefit 
from some accenting* method for portions of the programs-lines to serve 
as separators in. CMf editors or to box in accented words .for CAI pro- 
grams, for example.. Most of the low-costf alphanumeric terminals did. 
not provide line drawing capabilities, but some provide either two 
brightness levels, or the capability to reverse the character video 
for individual fields. Words, -sentences, or phrases can be putlnto 
foreground or background by using. one of the two methods. The terminals 
considered as candidates for the low-cost system have at least ope of 

these two features available. • 

* ** • " , ■* * •• 

All programs are expected to send an X-Y coordinate to positipn . 
the start of words -or phrases,' or to. echo characters back to the, user. 
Therefore, candidate terminals should have addressable cursors. "There 
is no standard practice for setting the X-Y position. Various sequences 
are "used to initialize the cursor setting mode, and different values * : 
are sent to the terminals *n either* X first or Y first order to set the 
coordinates. f . 

An additional requirement is that both alphanumeric and graphics 
terminals must have the ability to erase the screen on command fronr the 
computer and from the terminal user. All terminals reviewed nad bulk 
screen erase, van! most could erase, to the en/J of line or the end of the 
screen from* the cursor position. ;• -' ! 

* ' 4.2.2 Transmi ss i on" Speed 

A transmission speed of approximately 120 characters per 
second is. recommended when intermi xing 'upper and lower case alphabet 
and numbers. Transmission of ASCII codes over communication lines re- 
quires^the electronic formatting of the eight bit ASCII code between 
a start and a stop "bit. To transmit 120 .characters per second, a t*. 
least 1200 bits per second must be received at t»je terminal. *Al]~xer- 
minals considered for the low-cost system were able to receive data 
• *n a .switch-selectable range of 110 to 9200 or more 'bits per second, 
and all included the. 1200 bits per second rate. All the terminals 
included electronics to recede or send data using an RS-«f32C inter- * 
face" -to: a modem or data concentrator. » \ r 

4.2.3 Other Transmission Parameters 

To give the computer complete control over the characters 
returned to the terminal,/ and to provide for data Integrity/ of the 
user's input to the computer, full duplex transmission should be used. 
In full duplex, each character code is sent to the computer as soon- as 



a key has, been ^pressed*. The' computer stores the single key value and 
returns for display either the same code or .some other code which is 
program controlled. For example, during log on, a computer may not 
return the password for display byt may instead return a special char- 
acter. * ^ * * •* 
"-•>.' ■ 

Full duplex also, speeds up verification of data received by the 
computer. Half duplex transmission, not recommended for the low-cost 
system, customarily places the keyboard character on the screen 
'directly, and in parallel sends the character code to the computer. 
The computer does not return a character, so the user cannot verify 
that t(ie computer has received the transmission as. it was entered at 
the terminal, . * ' 

«■ • Full duplex character-at-a-time mode also implies that data are 
sent asynchronously, With no need "for synchronization with the network • 
processing unit* 

• 4.2.4 .Keyboard * ' 

* v . • .. \, . • 

All- terminals reviewed has a standard typewriter keyboard, 
with letters positioned in the standard QWERTY sequence. A special 
requirement noted for the low-cost CAI/CKI system programs is that a 
function key method 'should be available *tb send special non-display 
character commands to the central computer for CAI or CMI program or 
editor control. Most of the low-cost terminals include a numeric pad 
and some provide- a number of function keys . > Mds.t of Jthe keyboard' pro- 
vide a control key, so that special 1 *f unctions can be simulated by non- 
displayable control characters even if no function £eys are available. 

4.2.5 Interactive Terminal Selection 

V , ..... » 

Final selection of interactive alphanumeric terminals must % 
■ be based on specific us.er needs and applications. The described , 
functions are considered essential in any selection for. a low- cost CAI/ 
CMi'systcn. To configure an example low-cost system, the results from 
the foregoing survey were used in selecting candidate alphanumeric and 
graphics terminals wJyich^ineet the functional requirements. However, 
£hese are not the onl*y terminals which could satisfy the requirements. ~ 
The candidate terminals -for the example low-cost system are as follows*:" 

1. Alphanumeric text terminal for 'students, instructors, and" . ^ 
administrators— AODS> Regent" 103 terminal. ' . • - 

2. Text/limited resolution graphics/color terminal fer students 
Intecolor 8001 G terminal with lower case and function keys. 

< t * • • * 

3. - High resolution graphics terminal far student use— Tektronix 

4025 terminal with 0192 words of graphic memory. . ■ 



4.3 MANAGEMENT DEVICES 

• * \ 

A wide variety of devices can serve to -accompl i sh the # required CMI 
functions. In fact, $f keyboard input for data is sufficient and if 
hardcopy output is nof needed, the, student terminals can acpomplisb the 
tMI functions. The configuration of management devices which can s *be$t 
serve the needs of a low-cost system installation will depend on such • 
factors as the following: 

1. Types of input to be handled--manual input from Keyboard, 
pencil-marked forms, pencil-marked cards, badges, off-line 
response -devices, etc, ' '' 

2. Types of output required-rhardcopy output ; display at Inter- 
• active te/minal, specialized displays s etc^ / , 

'3. - Quantity of input and output to,be handled^-number of students, 
frequency of interaction, amount of information handled at each 
interaction, etc. - v 

4. Instructional and administrative decisions— procedures for 
registration, absence reporting* testing, prescribing in* 
structi on, managing student progress, etc. • 

In order to configure an example system without knowing the char- 
acteristics of the installation environment, it is assumed that 
capability approximately equal to the current AIS management devices 
will be required. Each of the management devices in the example -system 
will therefore consist^ of a forms reader, a line -printer, and the 
electronics to interface with the computer. HoweveV, the management- . 
devices jieeded fona specific installation site should be determined by 
analyzing the needs of that particular site. Considerable cost savings 
might be realized. if simpler management devices are adequate for the " 
CMI capability required at a site. 

4.4 COMMUNICATIONS NETWORK ' ■ * * 

• <■• ' 

The conmuVi feat ions network is considered to be the equipment . 
necessary to connect terminals to the communications processor. This 
section reviews various methods of transmitting data t;etween a host 
computer and Tone or more terminals at .distances up to 5 miles, the 
maximum expected between buildings for the low-cost CAI/C.MI system. It 
is assumed that dedicated lines would be used on an Air Force base. ' 

Equipment now available provides for^standard data format and 
allows for transmission of such "data over voice grade ^telephone lines. 
The ASCII protocol (line discipline, control code sequence or, data* . 
format) Has been widely accepted by government and industry as a* com-, 
munications standard. A system using modern data £ommuni cations 
formats and transmtssion media holds the promise of long range com- 
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payability and expandability between CPUs and terminals. k 

~. . The communications system for the low- tost C&I/CMI system must 
pr'ovjtfe for asynchronous full dup]ex data tnartsmissfon to each terminal 
at 120C bits per second (a* standard speed corresponding to 120 char- 
acters per v second] tfo allow for immediate character by character echo 
to the terminal display. Asynchronous operation describes another 
factor in -data transmission— the system can transfer data at the user's 
random rate; Full duplex describes the commupi cat ions system as having 
simultaneous communications both to and from the terminal, simitar to a 
* telephone conversation between two parti es . \ ' 

Modern approaches to data communication involve the use of readily 
available telephone circuits. The most promising options available are< V* 
to provide one or more dedicated line pairs from the central computer 
site to the terminals. Additional line amplifiers and correction of > 
phase differences are not required since the overall distance between 
the central computer and. any terminal, is expectej to be under 5 miles. 
Standard telephone pairs as presently in use with the MS system are • 
expected tr- be 'available for the low-cost CAI/CMI system. Full duplex 
operation requires a pair of two-conductor telephone lines to provide 
'a continuous communication linMn both directions between computer and 
terminals/ ' — ^ . • • 

The, two sections .following describe' line modulation techniques for * 
handling data between the central computer and the terminals. The • 
first section discusses single modems for communicating with single 
stand-al6!je terminals. TJie second discusses the recently 'developed 
statistical multiplexer systems which permit operation of multiple ter- 
minals on single communication fine pairs. 

4.4.1 Transnwssiort to Single Terminals 

>1ost host computer- related hardware can transmit data to 
terminals at distances up to about 200 feet. However, terminals 
located at greater distances should not be operated on lines with raw 
digital data; because of signal attenuation and susceptibility to noise. 
In order to transmit data to terminals at longer distances, modem' pairs 
are recommended. .Modems provide for conversion of the digital data 
by frequency shift or phase shift keying of an audio frequency. 
Applicable' types of modems include -trunk-line and short-haul types. 
Both type% provide the capabilities required for the low-cost Cfl/CMI 
system, including full duplex asynchronous modes, diagnostics, and 

interconnection to standard RS-232C electronic interfaces. 

* 

Short-haul modems generally accommodate a wide range ©f trans- 
mission speeds and do, therefore, contribute to a system's potential, 
for exapnsion. Speeds typically range from 1200 to 19,200 bits, per 
second and are switch selectable. The price range for short-haul 
modems is about $690 to $995 per unit. These modems can be used in 
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applications requiring high bit rates, such as some graphics appli- 
cations, and are good for distances up to 20 miles over unconditioned 
lines. X runtwl 1 n ^ modems are generally limited in their range^pf ^trans- 
mission speeds. This type of modem sells in the "range of $320 fo $525 
each, operates at dr below 2400 bits per second, is good for lo|g 
distances (up to several thousand nil es), but requires conditioned lines. 

The following modems were investigated to determine their capa- 
bilities for self-diagnosis in t he -event of »*sys tern- failure., ease and 
ability in matching the unit to the telephone lines, and ease and speed 
of maintenance, ATI of tfie listed modems 'are fuTl duplex, have asyn- 
chronous operation and RS-232C interfacing; provide diagnostics, and 
have a, minimum speed "of 1200 bits per second, 

TrunK Line Models • 

flarfufacturer Model. No. , . Speed (Baud) 

v ..... > _ 

Tineplex \ ' ^02 Up to 2000 

tele-Dynamics 7202T - Up to 1800* 

Syntech » . TT-202 Up to 1800 

Novation 202 * .1200/1800 

Penril ' 1800 Up to 1800 " 

. ' .... - 

i 4.4.2 Transmission to Multiple Terminals 

Statistical multiplexers (data concentrators) are available 
off-the-shelf. The units handle sampling biased on active users only, 
without wasting time segments on idle or, inactive terminals. The 
traffic demand is monitored' by a statistical algorithm and data are 
moved according to demand. A pair of inexpensive statistical mul ti- 
plexers can provide service over 2 telephone circuits to as many as 32 
terminals. - , , . 

Traditional error correction techniques can be applied with 
statistical multiplexing. In the event that an error occurs in the 
data format due to a disturbance in the communiqations sytem, the • 
statistical multiplexer will retransmit the data. The multiplexers 
can interrogate each other to repeat a- message to be sure tie message ' 
is transmi tted ^correctly. „. • J 

The statistical multiplexer approach utilizes a pair of statistical 
multiplexers (one at the. host computer and the other near the terminals), 
, two modems, and 1 a pair of two-conductor telephone lines. This con- 
figuration will typically handle four or more terminals. 

* . For* the statistical multiplexer approach ,* the modem paw r, the 
communi cation lines, and the multiplexer line ports are required to 
operate at only one-half the bit rate of the maximtfm composite terminal 
speed. When one to eight 120Q bits per second terminals are connected 
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to an eight channel statistical" multiplexer » for example, a channel 
speed of • only ~4W bits ; |>ef ; rstcoh<l (8'x 1200/2) is required.* The 
eight 'channel ^ statisticaT Wttplexer. system will operate in a degraded 
mode while five or. more terminals, are all receiving data at the saine 
'instant. Short-haul modems are preferred because* of their ability to 
operate at faster speedy at the flip of a switch. 

System expansion up to the capacity of the multiplexer involves 
considerably less cost than adding modem pairs as in the single terminal 
system. While it may be necessary ter purchase add-on channel cards -to- 
expand/ it Is not necessary to purchase modems and lines to service 



another terminal . 
expedted distances 
pi exer systems for 
maximum line spe&d 



For the low-cost 'system with relatively short 
from a central computer to terminals , only multi- . 
3 or 16 terminals have been considered so that the 
will not exceed 9600 bits per second. . 

. / ' • . t« * * i 

The statistical multiplexers and short-haul modems listed in the 
following tables were investigated to determine their self diagnostic 
capabilities, ease and speed of maintenance, and ability to easily match 
characteristics to the telephone lines. A pair of multiplexers antJ a , 
pair of modems are required for a system. The* first item in each list 
was selected ft>r the example low-cost system. - ... + 



Manufacturer 



Digitaf^ 

Communications 

Associates 

Infotrort^ 

Mi com 

Timeple'x 



Statistical Multiplexers 
Model No. 



Smart/Mux 115 
- Supermux 480 
*7 800 
M-8 



Max No. of 
Terminals 



16 
8 
8 
8 



Short Haul Modems 



v Manufacturer 



Ho. 



Syntech 
Tele-Dynamics 
Paradyne 
Coxtex 



L0M-7296 
7300 
SRM-192 
. 8200 LDSU 



5.1 SUPPORT SOFTWARE 



5.0 . SOFTWARE 



AND ANALYSES 




Support software can be defined as all -of the software supporting 
the operation of the\tow-cost system except the instructional (CAI and' 
CMI) software. . It includes the operating system, file mana'ger, loaders, 4 
linkers, additional compilers, assembler, and utility routines. 

Support software for one system is rarely executable on 'another 
system because 'of machine dependencies. Therefore, support software is 
not an area of system capability which should be evaluated in- isolation 
from the CPU. Rather, tne capability of support software is one * > 
♦iterion to be considered in selecting a .CPU. If possible, the W • 
inufacturer's support software should be used, instead of creating 
special purpose software pr modifying the manufacturer's* software. Thi,s 
advantageous for the fipl lowing reasons: 

1 - " ' ' ' '*.' ' ' * • 

The manufacturer 5 is obligated to perform maintenance on the 

support software, thereby relieving the user of this respon- 
sibility. *•'*•"♦ 

"Manufacturers often make improvements to their support software. 
* As reasonable requests from the user community are received, a 
manufacturers often implements $he enhancementsv and makes the 
improvements^ available to users. If the support software is 
not user-modified, new releases of software from the manu- 
facturer wi 1 1 not require modi f ication , thereby saving effort 
in acquiring upgraded support software. 

If the instructional software can execute without being 
Heavily dependent on special purpose software, advantage can 
be taken of advances m hardware technology and/or lowered 
hardware costs by readily transferring to an attractive new 
system. For example, if. the execution o,f the instructional , 
software requires the services of a special purpose operating k 
.system, then the instructional software is heavily dependent 
Sn .that operating system. Moreover, the instructional soft- 
ware cannot be readily transferred to a new system if the 
irget machine does not support the language in which the 
operating system Is coded. On the other hand, if* the execution 
enVi ronment provides s tandard i nterf aces to manufacturer ™~ 
- supplied operating systems, then the instructional software 
can\be readily transferred, to a new system. 

The support. software for the example CPU (the DEC VAX 11/780) was 
evaluated, and\ appears to have all of the capabilities required for the 
low-cost systeri It is therefore used* as. the support software for the 
example system configuration. 



5.2 PROGRAMMING LANGUAGE ,.. , . 

Evaluating a programming language to determine its meri't is a k sub- 
jective process. Factors entering into the evaluation of a language 
can be conflicting. For example., a language with many available v y 
features could be deemed powerful and therefore "good"— or, because of 
its many features, tne same language could be viewed as overly complex 
and verbose. The main difficulty is that a language is not "good" or 
, "bad" in. absolute terms. Previous work in programming' language eval- 
uation has shown that the merit of a programing language should be 
determined with respect to (a) the. application area for which it will' 
be used (in this; case, CMI and CAI application^) and (b) the computing, 
environment in which 1 the language will appear (the hardware equipment, 

the fiscal resources, and the personnel available).* ' 

>» * 
* ■ •• • •>. • 

»5.2.1 Top Level Language Goals , . ' 

^Definition of the goals for a programming language should * 
begin by considering the application area and previous research into 
what qualities make a language good. Then, by examining the stages of 
the program development process with respect to the application area, 
and identifying the most important objectives for a language at the 
various stages, the tqp, level goals^an be defined. .. . . 

In the CAI/CMI application asea, the stages of the program develop 
ment process are problem specification, design, coding, debugging, and 
maintaining. In this context, the following top lev.el §oals for the 
low-cost, CAI/CMI system programming language are established^ 

'. ' 1 

4. Ease of Writing CMI— Some capabilities are particularly * k . 
important in the design and coding stages of Chi programming.^ 
These include (a) allowing adequate record ami file handling 
capabilities for student data,.(b) supplying student pre- • 
scription and pacing constructs, and (c) providing Class- 
ification and typing of data and execution paths. 

* * . '* 

2. Ease of Writing CAI— Some capabilities are particularly im- 
portant in the design and coding stages of CAI programming. 
These include- (a) student response judging and analysis, (b) 
easily created and displayed screen frames, and (c) easily 
created questions and decision, branching. . . ' 

■ * i 

3. Rel i ability—The reliability goal is appl i cable during Coding , 
debugging, and maintaining. A language is reliable if it 4 
facilitates writing correct programs. Compile time checking, 

. execution time* checking, program testing features * and pro- 
gram debugging features are factors contributing to this |oal. 

* \ * » % 

4. Ease of Learning— Ease of learning is applicable to all the 

♦ ' stages of program development. It encompasses uniformity of 
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syntax and' context consistency. Unnecessary complexity of 
syntax and semantics should be avoided, A fairly common set 
■ of constructs, strould be utilized, supporting as few linguistic 
v concepts as possible. For example, x * 1 is a construct that 
, could also be used In the iterative construct ^ * ' 

' 'FOR x * FSTEP 1 TO 10 BO ( ). ^ : " 

. • • • * . > "• ' >r . . • * .* - , » ' \ 

the language sbould not contain Inconsistencies or context ' 
* dependencies (e.g. , a language that allows .int * 1 Where tnt 
.* is of Hype integer, but does not allow stng * "ABCQ" %nere 
,, sthg M * is of type stringi- i s i neonsf stent in the area of 
assignments). " *f % 

5. Maintainability— The maintainability, goal is applicable in the 
. debugging and maintaining stages. A language that supports 
. maintainabtllt^y provides readability* ease of source statement 
change* and - easily constructed -source statements i ? 
» . ; ■>•• .. ' " * '• :> .••• v ' ; ', 

ff. Efficiency—Efficiency is* relevant; in -the coding,' debugging', 
and maintaining stages. The efficiency m* language, can pe , • 
Measured* by comparing Jhe code' the compter generates (wjtn 
respect to storage and speed ) to thajf produced by a good 1 
^ ^assembly language programmer* . * » 



5.2.2 Cartdi date Languages * v , , 

Surveys of existing programming languages (Sammet, 1978; 
Brahan, 1973) were Used to determine the set of languages wb.ich might 
possibly meet the top levej language goals. ^Languages which could not 
possibly meet the goals (e.g., those designed for structural engrnr • 
eeri ng or 1 i n%ar p^ogr ammi ng ) . were el imi nated . Al so , 1 anguejes wrvtch 
might meet the go'als but were not current!^ implemented on at least ^ 
one computer were eliminated. Languages whi%have never been imple- 
mented, or were once implemented, but are no longer supported, were con- 
sidered to be too costly to make operational. The resulting set of > 
candidate languages is, listed below: « ^ . .V. v ^ 

ALGOL 68 - A powerful, general purpose, extensible language* used to f 
solve a large number Of progranlning problems. , , 

BASIC - A very simple l&nguage used, primarily in solving numerical 
problems, but with some advanced features. 

€AMlt - K general purpose interactive language, used for CAI/CWl,* -, 
A systems, and application, programming. v.* •/» 

COBOfc - An English-Tike language used primarily *or tt business data 
processing "problems*. * v / * . 

. \' ■ ' " •■ ■; . ... • ' ' r > 



C0RAL"66 - A general purpose language used primarily for solving 
al gebraic problems. . • 

COURSEftRITER' III - A simple language used for preparing CAI courses. 

fORTRAN - A language designed primarily for numerical computation -but 
j| used in, matiy other problem areas. ' •• % 

JOVIAL - A language designed primarily for numerical computations and 
data processing. * >V 

NATAL * A course-authoring language used in the preparation of CAI 
^ materials. ' * v . * 

PASCAL'* - A general purpose programming' language. 

PII2OT - A simp3e language for preparing CAI materials. , ' 

PLANIT - A language (embedded *i thin a system) for preparing and pre- . 
* * senti nf on-line individualized, instruction* 

* " * 

PL/ J - A general purpose language used to solve a large number of 
programming problems. % ~ 

TAL *:*- An authoring language used to present CAI materials. 

TUTOR -/i language used for preparing computer-based educational 
; materials. ' ' * . « 

5.?.3 Minimum Language Feature* . 

Language features were determined by examining the necessary 
CAI/CHIfuhctional capabilities from Appendix C and determining a set of 
suggested features for accomplishing those capabilities. Appendix § 
lists the CAI and CMI functions arid shows the language features to 
implement each function. From Appendix 6, it Js -possible to determine 
ta) which language features could be eliminated, as functional capa- v 
bi Titles are eVi mi nated or (b) -wMch functional capabilities might harve 
to^e eliminated as certain language features are eliminated. The 26 
lavage features are as follows. . . * _ 

1. "Built-in data types -of: '. ' 

i "INTEGER - e.g.., 1, 10,15 , 
REAL - e.g., 1.5,. 6.10 

LOGICAL - TRUE, FALSE , • - . • : 
W \fsmfm - e.g., "ABCDE" 
: ARRAY - e.g., ary$l ) = aryl(x) " 1" 
if RfCORi - e.g.*, rec.x»3 • ■* „ 

CLASS - e.g., dass(cl ,e2,c3). 



SET e.g., x 's setl* , m 
TEXT - long strings of textual data convenient fur represen- 
ting lange volumes of data (up to a screen page'). 

. Support of variable data - It is necessary to allow data to , • 
take on many values throughout 'the execution of a program. 
Many programs representing CAI/CHI functions need this eapa- 

• bility. An example from PASCAL is; „ . 

* * _ » . ^ • * * • * 

xT«10; . " ' / 

» . , . j, * „•••-• • 

• t * ,. ,"*,.*» 

X:?Y. * m 

. Support of constant data - It is necessary that some data 

remain constant throughout execution of a program. An example' t 
from CAMIL is: ' - v , 4 

♦ » • 

DEFINE INTEGER x-5; % cannot fee changed within this program. 

Explicit. dec! Ration of variables and constants - It is, desir- 
able that variables and constants- can be declared within, a pro- 
gram prior to use. This practice is commonly accepted as 
contributing to the creation of reliable software. An example 
from PL/ 1 is: •. , * 

DECLARE I FIXED INTEGER? , . r , - r \ 

k Definition of procedures (subroutines) and functions (.pro- 
cedures thaf return a value) - Capability to define procedures 
and functions is needed . This all ows the CAI/CMI programs .to 
be modular and therefore reltable and maintainable. An example 

* from PASCAL is:* 

PROCEDURE CALC <INVAL: INTEGER) i * ( 1 

BEGIN . . 

procedure body . ' \ 

V END. . / • " - " ; 

6. Specification of *iser-def ined datatypes - It is convenient to 

Y throughout, programs. This feature allows creation of different 
1 types 'of data beyond the built-in data types. An' example 

• 'from CAMIL is: ^ .« 

; : TYPE PACKED RECORD (INTEGER .1 j NUMBER N) RECTYPI. 

7. faciei ng, data wittiin records and arrays - It should .be possl 



V 



3\. 



to Specify packing data w1thin*records and arrays to preserve 
space. ( See feature § for an exalnple of a packed record. 

C. Automatic allocation and^access for Indexed Sequential files - 
.It should be possible to specify a .record oY group of records , 
to begin reading or writing by^supplyi ng key. for record •* 
positioning. This feature 1s necessary for efficient manip 4 * * 
ulation of student data on disk. ' . ••«■• , 

9> Automatic allocation and access for Direct fcccess;ft1es, - it* 
«' * should.be possible to position to *a particular record by 
specifyingL.its location on disk. This feature is necessary 
for efficient maintenance of student data on .disk. 

10 v File sharing - It should be possible for many processes, simul- 
taneous Tyactiye oft the CPU, to access -the same disk data file, 
v This feature is necessary for efficient manipulation of student 
' ' data on disk. . "\ . * * * * 

II,, ; Record reservation • It should be Jwssible for process to . 
lock .out at 1 .other processes whiTe iupdatt ng a student recbro/* 
pn disk^^This feature is necessary for reliable maintenance, 
of student data On disk. . v 

12. IF THEH ELSE construct.- Conditional, branching (cfioosing alter- 
nate program paths) should be possible, ..An example from: PASCAL^ 

is: . • .• . * ■ * * ' ' « '.. : » i " 

■••••-* . -ft 

. IF f X < > Y1THEN li-A ELSE ZVrB., , * 1. *V 

13. : Iterative statement cg»struct * Loops, should be .possible with- " ' 
P < in urograms using an iterati ve 'construct." Many of .the CAI/CMI 

functions -wi 1 1 be implemented with program loops.- An example 
from FORTRAN is: • . • /' 



.* , • . DO 10 K*1,10 1 
- >• !6op body ; v 

'» *•* * '*' "•« \ * ■ • 

; -— -."""-^'-jf^- , B ^ i ^ 'coiidTtitwa^- brd^ hing-- - It is necessary to be -able to 
*• choose one alternate out of many to implergent many of t?he 

f " » functions.- -An example from CAMIL is: " . 



CASE x OF - . „ . v r 

1 y «r 10; . 

2 z 15; ' ' " - ./ *■ • •. ' * v. . . 
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END. 

, * to . 

15. Assignment statement - It must be possible to assign values 

to variables wfttoin the language. This feature is. necessary i 
for virtually "all functions. An example from JOVIAL is: 

ai - ai +-i$. ; 

16. Program Interrupt detection - A construct is needed to allow 
for user Interruption of the normal program flow. :**Tlrt* featurfe 
is necessary for programs implementing CAT functions. An 
example from CAMIL'iS: 

ON Fl DO - 
BEGIN 



* % 



han^ the function key interrupt * 
EflD. 

17. GOTO* statement - An unconditional branching statement is needed- 

• *in the language. This is necessary to* transfer to specified 
sections of code. Am example from BASIC 1s: 

• ' . ' * . V «* 
GO TO 5. 

18. Conrounicatlon with CMI terminals' * The language must be capable 
of directing output to and receiving input from CMI terminals. 
Many CMI functions require this feature. k * 

* „ * 

'19. Communication with CAI .terminals - The language must be capable 
of directing output to and receiving Input from CAI terminal s. 
Many CAI function* require this feature.' 

• * * • . •' ■". * • ... 

2b. Arithmetic operations - Ttie, functions- of addition, subtraction, 
multiplication, di vision, Exponent jation, absolute value, * 
modulus, random. number generation, si ne^scosi-ne, tangent co- 
tangent, square 'root* string •concatenation, and .integer shift- 
*• ing should be available for uie in.CAI *mj CMI functions. An- 
example >fro» PASCAL is: • 



X*i» Y/2. 



• * iffl 

21. Relational operations - The operations of equal, not equal, less 
than , greater than, less than or equal , greater than or equal ? • . 
and set membership must be supported. Cp and CMI functions 

v • use these operations.- An example from CAMIL 1s: 

'.IF x <S y THEN z <- 10. 

- * - a 



22. Logical operations -The operations of not, and, or, exclusive 
•or, set complement, set intersection, and set union must be 
supported, for use in CAI and CHI functions, An example from 
ALGjQt#8 is: . 

IF J £ 10 /v Y £ 5 THEN lj «- 25. 

23; Interactive terminal page oriented textual communication - it 
must be possible to, display textual information by the^age 
at an interactive terminal via the -language. ' CAI functions 
. requi re that . text ' be d ispl ayed at . student termi nal s . 

24. Terminal graphics - A basic line drawing capability which can 
be utilized to construct terminal graphics should be resident 
within the language. .CAI functions may require this feature. 

. * • 

25. Hardcopy support - It must be possible to interface with a 
hardcopy printer* for generating, reports, program listings, and 

. other documents. CAI and CHI functions require this feature. 

26. Magnetic tape capability - It is* necessary to maintain .large 
volumes'.of student data on, an inexpensive backup medium. By 
using magnetic tape, £reas of mora expensive high-speed disk 
storage can be freed of student data; 

'5.2.4 Nontechnical Criteria 



While the technical merit of a language is of. piMmary* 
importance, certain non-technical .criteria should also bs,e considered in 



determining the cost *o? Implementing .the* OAJ/CHI functions in that 
language^. Tftese criteria are'Vs follows: * 




ngoag$ availability dn* recommended hardware - If a candidate, 
v language is not implemented on the host computer, there is a * 
' /cost associated with making the language's compiler operational 
on the system. • 

« * - . ■ "-•■«,.".*- . ' 

2. .^Quality pf candidate language documentation - This wi]l have a 
; K Bearing on 'how, quickly the software can be developed. ' 

% 3» Availability of trained progranroers - If there are no trained 
-'programmers available for a candidate language, the cost of 
fmplemen ting the software this^fcnguage will be higher. 

4. Availability of programs providing required CAI/CHI functional 
capabilities in $ candidate language If software to implement 
• CAI/CHI functioi&T capabilities already exists in a candidate 
language, then, with respect to this criterion, the cost of 
using that language will be decreased, • ! * 



5.2.5 Language Evaluations • 

To assist In evaluating candidate languages for the example 
low-cost system, an approach developed by Brosgol » Hartman, Nestor, 
Roth, and Uelssman (1977) can be utilized. This 1s an excellent, 
approach for forcing evaluators to think as- objectively as possible • 
about the subjective process of evaluating a programming language. In 
this approach, a fating matrix R» an application vector A, and an inter^ 
-mediate vector I are used in conjunction 'with a language, vector for each 
candidate language to k derive a technical merit score. The technical 
merit score and four management evaluation scores for a language are 
entered into a* language score vector which, in conjunction with a 
language management vector, yields a final score for each candidate 
language, the following paragraphs describe this approach, and suggest 
suitable parameters for use in evaluations for the low-cost system. . 

Matrix R is a 26 by 6 matrix whose rows represent the language , 
features (section 5.2.3) and whose columns represent the top level, 
-language goals (section '5.2.1). Each matrix element, Rij, is a 
numeric value indicating the extent to which language feature i con- 
tributes toward accomplishing language goal j. The application vector 
is a column vector of 6 entries representing the relative contributions 
of theSop level goals toward the CAI^CM^ application area. After deter- 
mining R and A, then R x A produces, a column vector I with 26 entries, 
each entry representing the contribution of a language feature toward the 
application area* '. 

•/Each entry. in the rating matrix R is a value between 0 and 10, 
indicating how much language feature 1 contributes toward accomplishing 
language goal j. The range 0 to 10 .is chosen* to allow the final tech- 
nical merit values to fall, in the range of 0 to ISO (the maximum 
technical merit score is actually 149.54). This rangecodd be greater * 
or less and still be representative of the technicj^HnSfit of a lan- 
guage. However, 0 to 150 is a reasonable range to work with, since 
technical merit values are then neither unduly large nor extremely 
small. The suggested, rating matrix R,,for evaluating languages for the 
low-cost system, is as follows: 
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The application vector A is a column vector of six entries, each 
entry value reflecting the contribution of a top level language goal 
toward £he CAI/CMI application area. The vector entries must sum to 
1.0, and reflet t the 'judged Importance of & goal with respect to the 
application area. Entries should avoid placing too much importance 
on potentially overlapping goals (e.g., reliability, leamapility, 
wrhTaTnaDTTr.ty ) .* * The recommended entries for appl t cation vector k 



are 



.18 
.18 

• rO 

.14 
.16 

.16 
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Since the entries in B represent the contributions of the language 
features toward each of the top level language goals, and since the 
entries in A represent the contributions of .each top leyel langyage r 
goal toward the CAI/CMI application area, then 8 x A produces a column 
vector I with entries representing the contributions of the language - 
.features toward -the appl j cation area . The product of the recommended 
vectors, R x A, is * \ " 



: .6.74 

: 7.96: 

: 4. ,12 

I "5.16 

: 7.38 

: 5.38 

: 5.66 



4.04 : 

. : 5.06 : . , - 

. - * : 5.40 : 

*■ . : 7.72 : • 

• - . : 7 .88. : 

;, 7.86 r : 

: 8.04 * a 

: 4.44 * : 

: 1.46 : . 

9 . : 6.00 : 

^ s , » 5 ♦ $6 * 

I"' . .:' •' 6.58 ; • ' : • 

- : 7.1£ ; 

: 6.04 , : 

: 5.98 # . * « 

: 3.32 : ■ - 

: 4.48 : 

: 4.00 : « 

A language vector CLi is then produced for .each candidate language. 
The entries in the language vector ;indirate3he^xfcent tolwhljh Jthe • 
1 anguage~"fe¥fures are. provTded. The^ product CLi x I will then produce 
a score indicating ^the technical merit of the language with respect to 
the CAI/CMI application area. These scores, ranking the technical 
merits of Candida te languages with respect to the CAf/CWt appl icatien 
area, can then be used in managenfent evaluations to arrive at a can- 
didate language recommendation. * f * 

The jth entry of language" vector* CLi indicates how candidate lan- 
guage'i contributes toward the jth language feature. For each language 
feature, the "degree of compliance" for a candidate language must be 
estimated." A suggested approach for making tmese estimates is as 



follows; " ; ' 

• T ; The language total ly meets the requirement. A score of 1.0 

will be recorded in the language vector. "'. . v" * 

P : The 'language partially meets the requirement. A score of from 
0.1 to 0.9 will be recorded in the language- vector according 
to the degree the requirement is partially fulfilled. 

. * .* . * . - • - 

F : The language fails to meet the requirement. A score of 0.0 

; will be recorded in the language vector. 

»■ ■ * ... 

, * . * - 

U : It is unknown from the available documentation whether the 
.., requirement is satisfied or not % A- score of 0.0 will be' 
: recorded in the language vector since poor documentation 

should.be considered even if the feature is implemented but 

not documented. 

After. Ct,i is established, the technical merit, TW, of a candidate 
language with respect to the CAI/CMI appl icatfoo "area can be, calculated: 

TMi CU x I. 

Development costs for tmpl enenti ng the CAI/jCMI software must be 
considered. Costs can be categorized as follows t 

1. Acquiring the implementation. language. 

• 2. Training the programmers in the given language. 
3. Designing the system. • 

• 4. Coding the system in the given ..language. . « 5 

, •* 

5. Testing, debugging, and verifying the system. 

6. Maintaining and upgrading the system." 

The entries in a language Management vector, LM, reflect the con* 
tri but ions of technical merit and of the nontechnical criteria to lower- 
ing system development costs. T|(e recommended LM vector is 



* • 

Technical merit : .33 : 

Language availability, on recommended hardware : .25. : 

Qualify of documentation * ' : .05 : 

Availability of trained programmers : .06 : 

Availability of instructional software in the language i .31 : 



A five-entry Language Score row vector* IS, pose entr jes^are 
scores ranging from 0 to 100, is established for each candidate lan- 
guage. .Each sentry in LS describes how well a candidate language is 
' Judged to contribute ^ 

is t$ie technical merit scores translated to a 0 

to TOO 'ttiti&f-lfcwifoBr' entries in IS are determined by examining a) 
the languages implemented oi the candidate hardware system (for purposes 
* of this example system, thelAX 11/780) , b) the available language 

documents, c) the availability of trained programmers, and d) the re- 
quired CAI/CHI functional capabilities for possible exlstingjinplemen- 
tation by candidate languages. This product LS x LM is the final score 
* for a candidate language. 

Clearly i sheeting ^ language for CAI/CMI application involves a 
considerable amount of subjectivity. However, if the foregoing pro- 

* cedures'are followed, the. evaluation process is made visible, the 
numbers entered into the process are open to inspection, and the , . 
result^ can be accepted or rejected on their merits. Also, the eval> 
uation process is designed to allow change to the vector values (the 
process could* easily be automated). In this way, opinions of uWo**ent 
individuals .can be considered before a final language recommendation is 
made. * • \ . 

* •# 

" In selecting a language for the low-cost system, the technical and 
nontechnical merits of the candidate languages should be considered In 
the context of the defined application area, The evaluation matrix 
* approach just described, or another explicit technique,. shouW be Jised 
to reflect the weightings assigned to the technical .and nontechnical 
merits of the languages under consideration. The actual selection must 
consider specific user applications, program costs, minimum^ language 
features, etc. Any language that meets these criteria may v be selected, 

/ft part of configuring an example low-c«st system, the technical 
merits of the candidate languages were calculated, relying in some wses 
on first-hand experience with a language and in other cases on published 
: . userUiterature. Ike nontechnical criteria discussed previously were 
also estimated, ending with calculations- of final scores for the can- 
didate languages. After application of the matrix approach and the 
parameters described above, CAMIL was selected for the example system 
configuratiom 
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5.3 APPLICATIONS SOFTWARE™ •* — — ■ 

• * | ■ •. 

5.3.1 Applications Programs Required . * 



' . T9 implement the CAI/CMI functions,, the following types of 
application software will be required: 4 

' .. , > 
1. CHI software — '. s 

: 0 Interface with the Interactive terminals. 

0 * V 

Interface with the management terminals. 

tfaiidfltiiHL.M._fQrffls that, are suhmitted. — 

• ° Scoring of tests (on-line and off-line). 

Resource management. 

0 * - * 

Assignment selection. * 

* Management of students to time targets. 
2> CAI software — 

CAI presentation program. * ' H \ • 

0 * * • 

CAI authoring system. 

0 Graphics editor. / 

3. * Editor capability 'to display and modify — 

° Course definition files, 
o 

Student data files. » * 



0 Test keys, 

0 Resource files. 

4. Report generation programs — 
o 



0 Training management reports a 



CAI lesson evaluation report. 
Course Evaluation Suimiary. / 

Si 

48. . * 
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Learning Center rosters, 



0 Studert block progress reports. . 

0 Absence report. % « 

' " ' * o » • • * ' 

Homework summary. • * 

0 A general .data retrieval and report program. 

5. Data base support software — 

0 An automatic CMI data base validation program. 

0 A program to remove student performance data from disk and 

• generate the tape and disk file for the generalized data 
retrieval program and for the Course Evaluation Summary. 

0 A program to remove the CALdata from the disk and generate 

• the tape files for the CAI report progyms. 

0 A program to remove test item data from the disk and generate 
the tapes used by the Test Item Evaluation report program. 

" 0 An archiver tol remove CAI lessons that are not in regular 
use from the on-line dat& base 4 and record them on taoe. , 
* . , .. »»• - 

0 A program to bring archived CAI programs back as thejr are 
needed. 

0 An archiver to take programs that are not tn regular use 
from disk and record, them on tape. 

0 A -program to bring archived programs back as they are needed,, 
5.3.2 Selection of Applications Programs \. 

* * 

A number of existing CBI systems were. surveyed to determine 
their capabilities in the applications programs area. All of the. 
existing systems have S0!9e„ elements ^ 

ized as cm and most have some CAI capability. All opthe.functionaT - 
capabilities required for the low-cost system haye beett implemented on . 
one or more of the existing systems, but no single existing system in-, 
-corporates all of the capabilities. • * 



A large proportion of the CAI and CMI functional capabilities for 
the low-cost system are currently Implemented in the set of app ications 
programs for the AIS. It is possible, with the addition of applications 



reared fw- the Tow-cost system The Survey of existing capabilities 
^Sf J? at J* wer ? r ?! rams have to be added to the,AIS soft- 
ware than to other existing systems. Therefore, the AIS applications 
software is selected for the example system. configuration, 

. 6.0 SYSTEM ANALYSIS 

Integrating the functional requirements with the hardware/software 
capabilities was approached by .determining the system requirements 
dictated by the full list of functions. Then, the additional system 
requirements dictated by the candidate support software and" programminc 
language were assessed, finally,* the total system requirements were 

SP^mS^/^ 1 !* 1 ** of tht ^andiclate -hardware to determine 
the feasibility of utilizing that hardware with the recommended soft- 
ware and language .-t» ea^llslrT^e^l^Tred CAI and CMI functions; 

- The (M/m functions and their payoff ratings were used asC m - 
starting point in determining the system requirements. An attempt was 
made to determine the costs, in terms of storage, I/O time, CPU time, 
or actual doliars^of each function. However, the functions are so 
heavily mterrelatSU that, .in Most cases, it- is hot possible to assign 
a cost to a single function. Fonexample, the information in a 
single file may service several functions. \ If any one of those 
functions is to tre included, that file must be included also, but the 
file can then be shared by many other functions without any additional 
cqst. In other words, costs are driven by groups of functions, and in 
general-, a cost will be incurred if any or all* of a group of functions 
are to b£ provided-.. Eliminating "one function from among a group does 
not usually result in decreased system cost." Therefore, in subsequent 
analyses,^ groups of functions are'-pe usual unit of. analysis rather 
than single functions, .However, if a, single function represents a 
considerable impact on the system, and can be isolated, then it is 
treated m that way. A list ^f the functions, with the files required 
and approximate CPU times Yieeded for each, is shown in Appendix <L 
These estimates derive Jrom measured times and known file requirements 
in the current AI§. The detailed requirements are summed and used as 
a basis for si fflttj^the.. system hardware^— — — - 

6T.V" DISK SPACE * . 

Required disk space can be estimated from* a knowledge of the files 
required, the required recor4 structures , and the number of records; 
needed. Experience on the AIS and the structures of the AIS courses 
were used as models in estimating storage space requirements. Para- 
meters .used were number Of courses, number of students per course, and 
numbers of blocks and lessons per course. Information on the config- 
uration of specific target courses was not available. Therefore, some 
assumptions consistent with possible application sites in Air Force " 
resident technical trains were made: 



1. |ive: courses 1«pl«wentpil on the system. 

2. One hundrei student per course- per shift (5p0 students ; on ... < 

CHI/CAI at "any one t}me>. ' , * . / / 

■ . * ; " 'S ' ' 

. 3V Five otbclts of Instruction per course, . 

4. Sixty lessons per ' Mock. ) > 

Some subparameters' within these maio^dings were also assumed, 
based an current practice: , 



1/ shifts, of operation. p 

■ *• »• • •••••• ■ „ 

2; , Students are assigned into learnt centers* with as.students 
i n each , c e nt er - r* — • 

' 3. ' Three course versions 

■ Implemented. \. ' ;* ^-..V ' • i^"-. % ' - 

five categories were used in estimating disk space requirements: 

'.i.'*.' . >; -\ -V".. . • ' . ' ■ ' 

J. Active programs. / 

2. . cal., - • • * • • .". ... - 

3. mi. < • . * ..,*•• t * 

*4 Recent uita file (RDf}» the pn-lW of ^most^recent 

X : student Completions of the blocks andrepurges. 

5. Support for the compiler, operating system, file management 
system, and the.neoe.ss.ary_supyort software.. / ^ 

The number of active programs, assuming *Hower softw^e i^tnten- 
ance currenVAlS, was es «^X5^»^&. 
60%^of current AIS usage, *for a total of J30 million bytes (3 bits per 

Disk retirements for t « ^programs were ^tima^uslng the^ 
current AIS structure as a aasis, but with a 75% safety margin.- Tim 
total then is 1 30 mil tj on uytes. 1 1 

The CMI requirements were also estimated Won the ^J^basis, but 
with a safety margin of 5056 .(CMI- requirements kati be predicted with 
, ^i SSl! ^ CHI requirement 1s 30 Si 11 ion bytes. ./ 

Tne Recent Data File requirement depends hon how ina ny student 
records thfusor activity wilT maintain on-ling. An est mated j£ 
million bytes will be required in order to maintain, on-line, the 

" ■ • * ■ . - '.K •. 



S^^^ri^ 20 S ^nte to vmUtrwh of the courses 
m(S eac P of the ^ocks assumed for the low-cost system.. > 

^ s ^i UP 2H t ^ r '** ^1^rthe'o^tiii9 ^'tlK file management 
^ the necessary support software will require 67 million bytes 
of 4isfc storage. This includes a 50% allowance for expansion should it 
^r^!: a ^\ t0 ex ?«nd the system s^r»«^l1tiSto^;v?d?%2 

:.J^ h ?^!I^^4? ^1tt^^..j^.4«^swy features;. 

§umrnai T y ' J*J tota1 requirement, for disk soace for the low-cost 
st^a e ge. aSSUnin9 U ° Studen H ,Vt * e RDF », 1s J* «*2s of / / 

6*2 TERMINALS » ' ' 

The assumptions that there wivi be 500 students on the svsten at 
any one time, and that 10* of th* instruction will be CAI lead to f 

i'I h€ CMI f u ^Horts; can bye accomplished Vfth a mix of on-line and 

rUlr^lJ^f^ te^^l: devices, but the retirements will depend • 

b^-^f S C K Se r / dpp11c ^ lons -' To ^put of f-1 invests . and ^provide 
printed, feeaback/prescription after each such input, a 1 oncost Winter 

site w??? hi SI%° r H°i^ r per1p 5 €ra1s - M ve such management terminal 
sites will be required to manage 500 students per shift 

■ ' .. >. s»i ' • • ' . . • ' ■ ■. ' • t .**' ■ " '; '• ■ 

"tyi.l^r W?AaSt inS 1 Wlfy terminals woul* be incorporated in a • 
trai«jng: 5 low-cost syste^fqr resident technical 

_ \ # Forty interactive •alphameric terminals for student use. 
•/•• ' 2 - Seven Tow ^solution .gra^ics/^ student use. 

~ 3, Three high resolutlon^raphics terminals for* student use. ' 
4. Nine i nterac ti ve al phanumeri C termi nat s for administrative ' 



use. 



•5. Pi ve management terminal sites. * 

" *■ i " , . , * ~ »• > ♦ " 

6tV_MEM0RV REQUIREMENTS 

, . Memory requirements are driven by such* aspects of the system as> 
characteristics of the operating system, number of terminals/ compu'ter 



;*rcnit«^ the : mm ^SSSS!^' ^ 

thrftmi of presewiatKms necessitated. by the funct ions affect the • 

representing the application |nrograii6 Wfrtch W«t I^B^Xfe- 
ooeratina concurrently on the system. In line with the design goals 
KsSptS 

(users at T 50 CAI terminals, *5mana^ and | admin- . 

istrative terminals). * w 

application processes oh the •^x^mcm^^.-m ^IJS/Sh 
shown' that processes similar to th*e representing the «PPUcttiM pro- 
grams implementing the r^c^ f "J^JL 
adequate but not ^tra^ 

response times) when 2^,600 to 3Q f 000 bytes of central memory 
allocated to each process, Therefore, .: the cental memory requirement 
due to the CAI/CMI functions is 1:9 million bytes, ; 

. Support for the compiler, the operating system* the "f«ti man age- 
Iment system, and the necessary support software ^ requires 100,000 bytes 
bf central memory^ Much of the operating system will continuously be 
in central memory, but t^he compiler, file management, and support soft- 
ware will be loaded only as required. * ' 

The total central memory requirement due to the CAI/CMI functions 
. and to' software support is 2 million bytes. 

6,4 DATA RATES 



The recorded functions (Appendix C) were reviewed,^w1thrn the 
framework of AIS experience, as to the signal ^^^^^1^ 
ments for the various' types of transactions, and the, rami mate page 
lengths to foe displayed. Integrating the* results 
rate of 1200 bits per second will be sufficient for most terminal dts- 
plays. Complicated graphics displays, depending on the ins^uctloj^l 
requirements and on the terminal capabilities, could* require a higher 
ra^.^refore,- an option for higher speeds is a factor for consider- 
ation in" the design of the system. 

j6.5 TAPE DRIVE REQUIREMENTS 

The number* of tape drives required is, determined by two ^ types of 
usaqe ! main shift and off-shift usage. Main shift usage will consis* 
of requests for data, submitted by course personnel , requests to restore 
archived' fesons, a*d requests tp restore arri ved programs , Jhe^re- , 
quests for data will include course evaluation summaries, te^t item 
evaluation reports, CAT reports-, and special retrieval requests. Most 
oTthlse totewH 1 be on tape. It is possible to f^his requirement 
with one tape drive:, but a tape drive is required for each process that 
is running! Two drives will allow two requests to be run at^he same 
. , " f -< « . ..... 

time.. . ♦ . • . 
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>• .k.* 1 !! 8 off - s ? lft J .»«S«'» s «»«BtsH*cl»ae batch jobs submitted by the bn- 
oata oases. Ine periodic jobs will, include the following: 4 

i: Remove the CM J summary data from disk and merge with the 
. history tapes. 

Z. Remove the CAI summary, data from disk" and merge with the 
<ni story tapes. \ 

' % 3 * tmT t6St Hm data ^ from ^sk Aftd merge with the history 

*• ^5 hi I? CAMjssiw that have not been used within a designated 
retention period. » * 

v * - " • * / * '■ 

. 5. Archive computer programs that have not been used' within a * 

designated retention period. . v M 

« *• . 

Although requirements could be met with one tape drive there until rt 
be severe penalties. These include: * there.would 

. 1 . Only one report from" tape could be run at a time; 

2. A scratch area on v a disk will be required to store data during 

3, Computer operators wil.l have substantially more loading and 
unloading of tapes from drives. • . ' I" 

If one drive breaks, no tape work can be done. 

Th^rmi*? 6r h* s yS»° reports can be processed at the same time, 
me CPU is normally faster than one tape oVive and can process data from 
one drive whie reading data from the Sther drive. Jhe P sysIIm Would 
aiso De able to go tape to memory to -tape,., with two tape drives With 

wh n f C h ta ?rfL v ^ iJ € t ? roc ? ss is ta > e t0 <w *> ^Kto^- 

which ^substantially .slower. Therefore, two tape drives are 
recoiwnended- for the low^cost CAI/CMI system. 

6.6 HARDCOPY PRINTOUT REQUIREMENTS / 

« *u Th f reports the five courses assumed to be imotementetf 

fJS? 1 T C °5 t 1 ? yStem . can be ***"y> on an of f-sh 1ft b£siT? by 

a medium speed line printer. At 600 lines per minute, production will 
be approximately 600^ pages of printout per hour. This is tore than S 
adequate -to sttpprt;-1ri#ermation retrieval and analysis requests, ' there- 
fore, a line printer capable of about 600 lines per mi nut* is 
recommended. 



7,0 CQNFXC9URATX0N OF AN EXAMPLE" SYSTEM * 

The preceding sect tons establish the hardware, software, and 
funetionaVjrequirements for the low-cost system, and designate candidate 
hardware and software Items for a low-cost system. The- foil owing para r 
^apfe&^coB system capable of meeting; the requirements 

and design goals as previously described. . 

7*1* HAINFRjlrtE AND PERIPHERALS \ \ , 

The example CPU, the DEC VAX 11/780, is capable of adequately but 
not extravagantly fulfiTling the requirements fir.$he loWHiwt CAl/CMI 
system. The example system will include mainframe and peripherals as 
follows; , 

' Monthly - 
Maintenance 



One standard 

VAX TT/780 package- 

with': • ~ w 

- • ^ : * ' 

♦ 1 . 51 2K bytes of memory. 
•2. One REP06 17 ; 6 million 

byte disk drive with 

MASSBUS adapter. 
. 3.. One TEE1 6 .45 inch/ second 

tape transport with « 

MASSBUS adapter. 

4. One QZ11-A asynchronous 
multiplexer for 8 lines. * 

5. One LA35 OECVriter M 

console. $185*000, . $827; 

Additional equipment to f 
meet hardware requirements: 

1/ One MS730DB 512K 
i „ byte memory expansion. .£2,000 , 110 

2. One MS780DC 1,024K 

byte memory expansion. 35,000 * 220 

3. One REM03-AA 67 million 

byte .disk unit •« 

— wttH "Mft^SBWS adapteirv * 25, 000 T70 

4. 0ne*TE16-AE 45 inch/ V 
second tape transport. 31*290- &n 

5. One LP! 1 -DA 96 character 

6j60 Tine/minute printer. 25,700 . * 185 

6. '.One DZ11-B asynchronous . ... 

multiplexer for 8. lines. . ' 1,800- 21 



Monthly 

-rCost, Maintenance 

7. Three DZ11-E asynchronous 
muUtplexers for 16 lines* * 

. at $3,860. ' $-11 ,550 • • $ 75 

8. One H9602-DF UNIBUS ... V 
options cabinet. : v 2 r 300 «• none 



" ^ Totals $319^640 * $1,668 : 

7.2 k TERMINALS * 

The Atypical w system configuration (section 6.1.2) includes 9 
administrative terminals, 40 alphanumeric text terminals, 7 low 
resolution graphics/ col or terminals, 3 high resolution graphics 
terminals, and 5 management terminal sites." If the administrative 
terminals are the same as the stjjdeni '..text terminal s.(a feature that is 
recommended to facilitate substitutions in case of faiwre), terminal 
requirements become (a) 49 alphanumeric terminals, (b) 7 limited 
graphics/color terminals, (c) 3 high resolution graphics, terminals, 
and (d) 5 management terminal sites. 

From section 4.2.5, the example alphanumeric text terminal is the 
ADDS Regent 100 with function key option*. The example low resolution 
graphics /col or terminal n s the Intecolor 80016 witn lower case and 
function keys. The example high resolution graphics terminal is the 
Tektronix 4025 with 8192 words of graphics memory. The costs for stu- 
dent and administrative terminals for the "typical f configuration are: 

49 ADDS Regent TOO at $1100 * $53,900 

p 

7 Intecolor 8001 G at $2700 » $18,900 
3 Tektronix 4025 at $4495 - $13,485 



. $86,285 

t , For purposes of configuring an example system* it is assumed that 
each of the five management terminal sites will consist of an OPSCAN 17 
optical forms reader, ~a Teletype Mo^l 40-ROP printer, a 
raent Corporation LSl-ll mini -computer with peripheral cards, and various 
displays, push buttons, and specially designed circuitry. Each such 
terminal will cost approximately $26,000. Total cost for the five 
management terminal sites wi TV be: 



5 Special Management Terminal"^ at $26,000 * $130,000. 



7.3 COMMUNICATION* SYSTEM . 

* A 64-port system for 59 student and administrative terminal's and 
five management terminal sites is required for the example system. It *■ 
is assumed that four buildings : (wk* areas) will be used in the -oper- 
ational Tow-cost system, with the computer and some classroom space ; v 
in one of the buildings, to provide communications. to the three remote 
buildings, three statistical multiplexer systems, each handling 16 . 
terminals, will be used. The computer ouil ^^ding i§ assumed to have 16 
terminals connected directly to the network processing equipment of the 
computer. " •* ■ - ' 

The configuration to each of the remote buiWinds will include a 
pair of 16-channel statistical multiplexirs" (one at the host computer 
and one in the remote building connected to the terminals), a pair -of 
short-haul modems (one at each multiplexer) operating at 9600 bits per 
second, and two twisted pair telephone lines. The entire communications 
configuration will then consist of six statistical multiplexers, six 
short-haul modems, and six pairs of Government furnished, twisted pair, 
dedicated telephone lines. Using the Digital Communications Associates 
(DCA) Smart/Mux 115 statistical multiplexers and the Snytech LDM-7296 
short-haul, modems (section 4.4.2), the cost of |he example commumcation 
system Is: ,\ : 

6 DCA Smart/Mux 115 at $4300 = $25,800 
6 Syntech LDM-7296 at.$ 690 - $ 4,140 



$29,940 

* 

7.4 LIFE CYCLE COSTS I , 

The initial cost of the hardware represents only part of the total 
cost. Some changes in the facilities, Organization, and courseware Jill 
'add to the) initial cost.. During the lifetime of tbe r system, costs Will 
be incurred for maintenance an4 supoort, mater ials* and supplies, energy 
used, and initial, as well' as some continuing, training for personnel 
using the system. " 

Eight years is used as the total life of the system. Costs for 
hardware and software .maintenance and support are estimated and 
escalated for those 8 years. Some costs could not be established 
because o* lack of required information. For' example, courseware costs 
xannot be 'established unless detai led information about the target 
courses is known. Also, costs per work hour cannot be established With- 
out details of the internal costs of the organization furnishing the 
manpower. In these cases, then, a work hour estimate without dollar 
Costs "was established. * • 

The following paragraphs indicate how the calculations were carried 
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out, awl show estimates in work hours, for the identified tasks .5 Where 
appropriate, quantity discounts (over 10 units) are included. However, 
savings from original equipment manufacturer (OEM) agreements, which 
might be effected on all equipment, are not included'. • ' 

7*4.1 Mainframe and Peripheral Hardware t 

Initial costs of the computer mainframe and peripherals (not 
including terminals and communicatio/i e%iipment) are listed below. 
Maintenance costs for* successive years aye calculated by increasing each 
previous year's cost by 65&, the percentage of increase projected by 
Digital Equipment Corporation for salary and spare parts, increases. 



Initial Cost (Section 7.1 above)\ $319,640 

Year 1 maintenance cost 20,016 

Year 2 maintenance cost 21,216 

Year 3 maintenance cost *? 22,489 

Year 4 maintenance cost ^ 23,839 

Year 5 maintenance cost + * 25,269 

Year 6 maintenance cost , - , 26,785 

^fmr : 7 maintenance cost 28,392 

Year 8 maintenance cost 30,095 



Total 8 year cost v $51 7,741 

♦ 7.4.2 Software Development 

* The two main areas of software development for the low-cost 
system are the transportability of the A IS developed software and the 
additional appl i cations pifbgrams necessary to meet the functi onal 
requirements of th# low- cost system. The transporta&iTlty of the AI5 
CAMIL software is presently .being carried out by the McDonnell Douglas 
Astronautics Company. The additional applications software is estimated 
to represent three work .years of effort. ' This figure includes all 
functions recommended for .the Tow-cost system (Appendix C) and not. pro- 
vided by current AIS applications software, except interfacing with 
other Air Force management systems. 

' 7,4.3 Software Maintenance 

: V For purposes of estimating the cost of maintaining the soft- 

ware of the example system, one fuTl time appl ication programmer and one 
W1 time system programmer are included. The maintenance wi IT include 
minor improvements to tne software as wel l a s corrections to faulty 
software ftfue to programming errors in the initial software). 

- Should expansion of the recommended system occur at one Air Force 
base, the two software maintenance personnel described in the preceding 
paragraph could maintain four systems/ Because the additional systems 



ar* duplicates of the Initial system, -changes or corrections made to 
one can easily be applied to the remaining systems. . 

If multiple versions of the initial system should be implemented at 
various bases, additional software maintenance personnel would be re- 
quired. |wo iTOintenance personnel with* necess^at^^ni^al 
base, as described above, with one additional application programmer at 
each additional base to implement the modifications determined by the , 
personnel at the Initial base. 

* > 7.4.4. Operations 

When hundreds of students and instructors enter data into a 
computer, ,, fouT-ups w are inevitable. This, plus software and hardware 
failures, establishes a need for monitoring. A1S experience confirms v 
the need for monitors to be, located in the classrooms near the terminal 
areas and in the computer room. * 

One person with a thorough knowledge of the operation of the , 
computer system, stationed near- the terminals, can greatly improve the, 
operation of the system and save on Instructor hours. This person 
""should be avail abW at the terminal for correcting minor student/ ♦ 
.computer interface problems,; This frees the instructor JbM™* 3 * 
student's instructional problems. This person also is responsible for 
Seeping the terminals in' working order, and for calling maintenance . 
witt any of the more difficult operational problems. The number of 
peHfons required for this task depends on % the locations of the , 
terminals. If several management terminals are located near each other, 
one person can'handle problem*!* several terminals. As physical 
separation grows, additional people are needed. This report assumes 
that the user group will furnish these personnel and that they will be 
trained as needed (section 7.4.10). 

$ Computer operators are needed at central site to toaj tapes, . 
answer maintenance <*al|s, keep records, and generally >nomtor computer • 
operations. Since Sis*system is a prodfuctvon system, K ^ is expected 
that operator intervention will be minimal . Limiting batch Jobs and w 
other tasks that require operator intervention *o specified times of 
day- further minimizes the manpower requirement. The effort per system 
can be minimized if the mainframes for several low-cost, systems are _ 
located near each other. For. cost estimation purposes,; pne operator per 
- system per shift and one additional backuD^pera tor are included. • 

7.4,5 Terminal and Communication Equipment 



, The terminals mi coiaminications-.eduipmeht •• requi red ^for a 
"typical" low-cost CM/ CHI system configuration are as follows \spare 
parts requirements are listed in section 7.4.6): . » 
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49 » ADDS Regent TOO at $ 1100 = $ 53,900 
7 Intecolor 80016 * at $ 2700 ■ $ 18,900 
3 Tektronix 4025 at $ 4495 ■ $ 13,485 
6 DCA Swart/Mux 115 at $ 4300 * $ 25,800 
6 Syntech tDM-7296 %t $ 690 * $ 4,140 
5 Management Terminals at $26000 - $130,000 * 

Total Purchase Price * $246,225 

7.4 J ^Tlrmlnal and ^mmunf cation Equipment Maintenance , 

Because* of the proliferation of equipment from different 
vendors, it is not practical to procure maintenance contracts from each 
'individual vendor. Most manufacturers require/ that malfunctioning* 
equipment be delivered to their own service centers. The manufacturer- 
service approach generally fieans; ; ** 

1. High cos Jt per junit. maintenance 'con tracts (usually 3/4X^0 1* 
of purchase price per month). 

5 2. Shipping or travel time delays. 

3. A requirement for many complete unit spares. 

The recoi^fended maintenance approach is to provide an on-site / 
repair Capability for non-jaainframe equipment. The primary duty of \ 
these personnel -is to maintain individual CRT terminals* cowmmicawn 
equipment, and management terminal peripheral equipment (such as the 
forms reade/, controller, and printer) by troubleshooting the equipment 
anl replacing malfunctioning printed circuit boards or individual com- 
ponents. /Secondary duties include accenting trouble calls- from class- 
rooms*, arfi Seating and replacing malfunctioning equipment, for cost 
estimation purposes, 4 work years per year are included for these tasks. 
■I ' ■ ' \ • - . ' "/ ' 

>. Using the on-site maintenance personnel approach', a supply of 
spare parts is needed for CRT terminals, communication equipment, and 
management terminal component equipment such as mark readers , con-* v 
trollert, and printers. To ccyver spare parts for all equipment except 
the printers, a spare parts budget of J% of purchase price per year is 
called for; AIS experience Indicates Chat the heavily used mechanical 
IL*?]!*;? reguiYe a 25% per year budget, since the entire or Inter must, 
be replaced approximately every 4 years. 

4 Using a rate of 7% of purchase price per year for a spare parts 
budget for non%ia1nframe equipment except management teminal p¥i|niers» 
Which are figured at a 25% annual budget, the first year costs are as 
follows: , s 




m m& Regent TOO at $ 77 * $ 3,773 
7 Intecolor 80O1S at $ 
3 Tektronix 4025 at $ 315 - $ $45 

6 DCA Smart/Hux 115. at $ 301 * $ 1,806 
6 Syntech LDJ1-7296 at $ 48 - $ 288 
5 Management ^erminals— 
Non-printer equipment at $1540,= $ 7,700 

J Printers (25* rate)' at $1000 » $ 5,000 



First Year Spare Parts Cost 
* of Non-flainframe Peripherals $20,835 

v Using an appreciation rate of 7.5% per year for the increase in 
cost of repUcement parts, the spare parts budget for the first 8 years 
would be as follows: *. s . , 

For year 1 , the spafe parts cost is $20,835 * 

For year 2, the spare parts cost 1£ ^22,398 * y \ 

For year 3,' the spare parts cost is $24J)78 

For year 4, thje spare parts cost is |26,676 

For year 5, the spare parts cost is $28,677 

For year 6,. the spire parts cost is $30,828 

For year 7, -the spare parts cost is $33,140 

For year 8, the spare parts cost is $35,625 , 

Total spare par^ajpreciated cost is $222,257 
\. 7.4.7 Facility Requirements w '* •...« \/^ - 

• Installing a computerized training s|stem in a resident 
technical draining environment sometimes necessitates facility changes. 
The changes that are necessary; depend of course, Jn ^ fisting v 
physical plant, computer locatt on, amount of mediated mtf ^U *«* m W 
other factors . Without known parameters,, it can only be sta|ed nere + 
that m W .^iuatton 1s large learning centers located close to the 
ranaaement terminals and as Close as possible to the- computer room. 
1?S of colter loom needed .for'thfsjnstallatio^ls J[ 
350 square feet. 55 The' computer room must be a*r conditioned to maintain 
approximately 70 degrees F With a 48,900 BTU per hour load. 

7^8 ^Energy Requirements . - , 

The energy requirement for the system consist of the direct 
electrical consumption for mainframe, peri pnerals, and *ejjn»»5. \f ■ % 
the air conditioning for the computer room. Tne total direct mctrica} 
power requirement is 29 m per operating hoyr. The air conditioning . 
requirement is for .48,000 BTU per hour. s . .... • / * 
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7.4,9 Courseware Development 



'.„• introducing > CBI system into an on-going course of ins true* 
tion will, almost 'always, entail costs for changing courseware. These - 
changes will vary greatly depending on the existing environment. For 
example, an extremely important factor t$ whether course is' currently" 
self-paced. Without definition of target courses, these costs have not 
seen estimated. • • * 

7.4.10 Training " V . • t . \ t 

\- The success of any system is largely dependent upon the * 

sopbisti.catiow of its users. To this end, training costs sWra be in- ' 
eluded in the life cycle costs. Experience. with the operation of the 
AIj has identified several categories of training thai are highly 
recommended. The Jevels of training and the estimated hours of instruc- 
tion are as follows: 

1. The Deputy Cowmander for Training (TTC/TT) and subordinate 
divisions, e.g., the Plans and Requirements Division (TTCX), 
constitute the upper level of local training administration ' 

, anc management. Training at this level should consist of a • 
System overview- and will \require approximately 6 instructional 
hours. Technical Training Group Commanders and personnel of 
their Resource Management Sections (TTCC) should alio receive . 
training at least equal to that provided TTC/TT. 

» • . « 

2. Branch 'Chiefs and course supervisors are, the next level of 
training management and- shduld receive a wore complete pre-, 
sentation of the aspects of the system. This level will require 

* s approximately 60 instructional hours,. 

3. Curricula and Instruction and Measurement personnel should 
receive training in- the evaluation aspects of the system. This 

* training will require approximately 120 instructional hours. 

.„., 4. ^ Instructor supervisors and instructors should be trained in 
the operational aspects of the system. The instructor super- 
visors should receive approximately 9 instructional hours, and 
the instructors should receive approximately 80 hours of f 
instruction. » 

> 

5. Course Data Base Manager and Student Control personnel should 
receive instruction on manipulating the course data base to fit 
course organization and neeos*. The data manager will be 
expected to make all data base changes and should receive 120 
hours of training. The student control person will work 
directly in the day-to-day operations with the students and 
should receive 18 hours of training. 

\ 



6; School Data Base Management personnel shoul d receive more 
global training/ becoming familiar with all^ourses and pro- 
viding leadership* and services fn'such areas as strategies, 
student management, documentation training, and course 
liajson. This "training should be approximately 400 hours, 

: * 7:4.11 Supplies' . \ * ' 

The necessary consumable supplies, used mainly with tlie •' 
management terminals and off- line 'testing, are: (a) central sitelprinter 
paper, (b*) management terminal printer paper, (c) student test forms, 
and (d) miscellaneous supplies, such as printer .ribbons, computer tapes, 
disk packs, tools, and cleaning supplies. Estimates of these costs are 
based upon current experience with AIS in usage per on-line student* 
The number of students on-line is assumed to be 500 per shift with two 
shifts. The estimated costs of these supplies are based on the following 
assumptions: 

1. Test, form usage is estimated based on the assumption of five 
transactions/student/ shi ft. Total forms usage is then calcu- 
lated by assuming 250 training days per year, 'with 30$ waste 
and failure. Total yearly cost: $19,500. 

2. Management terminal printer- paper costs are estimated by using 
the five transactions/shift fvgure. . The yearly, cost: $6,500. 

3. Central site printer* paper usage is estimated by assuming 'that 
a production system will not have software development. Usage 

* will include CMI reports, CAI authoring and evaluation, course 
materials development, and other administrative and evaluation 
usage. Usage is estimated at 30% of current AIS usage. Total 
yearly cost: $6,000. , 

4. Magnetic tapes and disk packs will be needed at the central 
site for data storage ana backup. It is estimated that $500 per 
year will be required for tapes. Four extra disk packs should 
be purchased the first year and an average of one per year 
thereafter. Toese costs .-are detailed below. 

1st year = $2,700 

2nd through 8th year - $675 jper year 

5. Miscellaneous supplies include needs for day-to-day operations 
such as tools, bench stock, cleaning materials, print% ribbons, 
etc. The estimated yearly miscellaneous supply cost: |$1 ,500. 

The summary of supply costs, escalatea at 7*5% per year, i| shown 
below. * • ... 
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1st year $36,200 

■-"V" 2ntf~year $38,915 - <v. • — - — 

*' / 3rd yea r $41,833 V , •? 

4th year $44,970 ' 

5th year $48,342 • 4' . 
6th ye^tr $51 ,967 v 

7th year $55,864 1 - 

8th- year $60,053 > ; * 

' • * TOTAL for p years » $378,1 44 v . 

7*4.12 Cost Sunmary _ V 

Total \ , Recurring 

8 Year A t Costs, - 

Paragraph/Title Cost For A / Per Year 

One System \ f ' 



7.4.1 Mainframe* & Peripheral Hardware $517,741 \ 

7.4.2 Software Beve1o|)ment ^ 3 MY \ ' ' 
7*4.3 Software Maintenance 16 MY* \ \\Z MY* 

7.4.4 Operations > 16 MY* A \2 MY* 

7.4.5 Terminal ^Communication „ .. I 
Equipment v $246,225 » \ 

7.4.6 TerminaT & Communication • V 
Maintenance . / $222,257 S27,782 

% ; 32 MY " v>. 4my 

. 7.4.7 Facility Requirements 350 sq. ft. r . 

•for Computer Room 

- 7.4. 8 f Energy Requirements, v 928 MWH 116 MWH* 

two shift operation / 1536 MBTU 1§2 MBTU 

7.4.9 Course Development ^_ Unknown •-%£■-. 

7.4.10 Training - Unknown** — »«— 

7.4.11 Supplies ' * $378,144 , $47,268 

, — , — _ ....... ; . <3 - • . i . ■ • 

SYSTEM LIFE CYCLE COSTS: 

Hardware & Supplies $1,364,367 $75,050 

; ; : Manpower , ' . . {% 67 MY 8 MY , 



i* Additional sys terns », if co- located, with a first system, could -s£ar& 
personnel . * , * - 

** Required hours will depend on personnel turnover rates. 

8.0 CONCLUSIONS 

The principal cpmjpnents of a dedicated local system will include 

% A^c. % - * / "A . 'A 

.? Computer and peripherals. 

Student and administrative terminals-*a1phanumer1cs, graphics/ 



rt <»Tor, and high resoUitiqp graphics. ;i 

° Management device— farms reader, printer, and controller. 

° ComnmnicaXions network, 

0 Software. 



An operationally configured dedicated local systems to support 500 
students 'per s,hift on CWI and CAI in resident technical training, can be 
acquired. for\ approximately $500,000 (197$). * : * 
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APPENDIX A 

LISTING if CAI/CMI FUNCTIONS FOR SURVEY 

I . STUDENT TESTING AND j| VALUATION - Usi ng the computer to assist the 
instructor in evaluating the student. The tests which are read and 
scored* by the computer Ian include, for example, aptitude tests, lesson 
or block tests, perforAice checks, attitude measures, or biographical 
.information. Questioner could be true-false, multiple choice, or con- 
structed response (single words, or phrases, or sentences). Students 
receive quick feedback, Instructors don't have to score tests or 
record grades, and all test results are readily available for reports 

or evaluations. . . ' • - 

• , * k • • • 

\ A. test Scoring and Feedback CapabiTi ties - The computer system 
can seore those tests for which test keys have been previously 



The results from ea^ch test- can be printed out for the , 
student and instructor. Questions are true-false or multiple 

choice. „" . i% * K 

RATING: * 

- "* 

1. Answers are scored right or wrong, and a designated percent 
correct is .required to pass the testT RATING: 

2. Tetal score on the test is Corrected for guessing.. 

RATING:.' 



3." Some questions can be weighted more heavily than others. 
4 RATING: 



4. .'A designated percentage of 'a set of objectives must be 
passed in order to pass the test, for example 3 out of 5 
objectives. « - .RATING: 



5. A specific. objective or objectives must be passed in order 
to pass the test. - RATING: 

I * . .,- • ' 

6. A specific question must be passed in order to* pass the 
objective (or the test). RATING: , 



7. Performance checklists, «s completed by the instructor, can 
be scored by the computer. i&* RATING: . _ 

8. An instructor can bypass: a test by "certifying* that the 
student knows the material. ' RATING: 



B. Ad ditional Qn-Line Testing Capabilities, - ; §|udents can be tested 
bh-1 ine at a computer' terminal . The order of questions and the 
. sequence of alternate answers for each question can be scrambled 
automatically for each test administration. On retestirtg after 



an initial failure, retest can be on only the failed objectives. 
Questions can be true.rfalse, multiple choice, or cohstmicted • 
response (English words). The amount of testing, can be depen- 
dent on the student's earlier performance in the course or on 
this test. ' 1 RATING: . 



1 .* .Random presentation order for questions . s RATING: 



2. Random presentation order of alternative- 
answers. ' RATING: 

3. Retesting is only over failed objectives. • RATING: 

., • • j» ' • ... » 

4. * Questions can require constructed response 
^(English) answers. / * » RATING: 

5. Amount of testing depends on performance, on 
earlier questions in this test. RATING: 

6. Amqynt of testing depends on performance" 

earlier in the course. * RATING :\ 



C. Determination of Pre-Course Student Characteristics— Provides 
for use of Student -data pollected before or at the beginning of 
training. Uses can* 'include, for example, examining eliminations 
as a function of aptitude and years of school or prediction of how 
fast a student will complete the course. * RATING:, 

). Computer stora|e^and retrieval of relevant Air Force 

personnel data - for* example,- ASVA& or AFQT scores, 

or biographical data such as previous schooling. 

. .« • . < RATING: 

■ r '. 

2. Storage and Retrieval of Student's Prior Knowledge 
I of Course * Block, or Lesson Objectives scores on 
4 criterion-referenced pretests on other measures of 

training objectives.' v »~ RATING: 



"""3. Storage and Retrieval of Student's Learning Process Skills 
for example, any ability, attitude .or interest type measures 
such as reading comprehension, study habits, etc. 

* k RATING: 

Storage an& Retrieval of iStadent's Course-SpeiCific Critical 
Entry Skills - for example, aptitudes,, abilities and inter- 
ests that are specifically related to .success in a par- 
ticular training specialty, such as mathematical ability 
or interest Tin the career field. V: V - RATING:" 



... r 



D. Determination of WithiR-Course' Student Attitudes and Interests 
Provides for use of attitude of interest data collected tn the 
•course of training. Uses could include evaluation of instruc-. 
, tional materials or procedures, prediction of completion times 
or rates of -progress, and prediction of field performance. ■> 

s - mum; 

1. Storage and Retrieval of Student's Pre- and 'Post-Course . 
? ' Attitudes, - for example, student's attitudes toward 

training,' materials, or methods. RATING; 



2. ' Storage and, Retrieval of. Changes in Interests- or Attitudes* 
During Training - for example, periodic questionnaires to 
- assess. interests in the training itself,, or the media. 
§ " , RATING": -■ 



II., STUDENT ASSIGNMENT t-IN SELF-PACED COURSES) - Use, of the computer to 
assign trainees to lessons, tests, or other instruct ionaj alternatives. ..a 
The assignments can be l>ased on considerations of lesson content ^avail- 
able resources, student characteristics, or student performartce?,datau • 
Students receive their, next assignment immediately after completing 
their preceding lesson, and training resources are scheduled for 
optimal usa<je. . , . . - - 

A. Assignment to Alternative. Course Versions - Cdurse versions 
whf<h differ as to content (perhaps for % different sJ?redou£s of 
J * a specialty", or to test and implement new, training materials) 
can be established. A student .enrolTed in a particular course 
version will be assigned only the materials for that version. 
■ ; • >•<•■ . RATING; * 



8. 



Determination of the Order of Assignments - The computer takes 
'into account various classes of information in determining what 
to assign next to a trainee. The information considered can 
v include j, for example, work the trainee has completed to date, 
the trainee's rate Bf "orogress to date, availability of instruc- 
tional resources, and requirements, to assemble a team of * % , 
trainees foh a training tasi<, ~ t RATING: 

1. Order of assignments based on lessons completed to date. 

* * • . RATING: * 



2, Qrder of assignments based on performance (time and 
scores) on earl X&r lessons. , ^ RATING: 

3. Order of assignments ibased on avail ability of resources. 

i , - ,,. RATING; 



4. Order of "assignments based on considerations of assembling 
a team of trainees for a team task, - RATING: 
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5. Order of assignments determined by allowing instructor to 
pick from available .lessons. RATING: _ 

Assignment to Alternate Less ons - Provides for student assign- 
ment to .a vai 1 able al ternate medi a (e.g., printed vs. audio- 
visual media) or* alternate versions (e.g.. easy versus difficult). 

. . RATING: • 

, . , ' ' 5 

1. Based on Instructor Selection - instructors select alter- 
nates based on their assessment of the student's needs. 

RATING: ' 

* . 1 1 * ■ in n ' l » 

2. Based on Student Selection -^sttiflents select, fr&m lists of 
available alternate media or lessons, the particular one(s) 

* they "prefer. • RATING: u 

3. Based on Resource Availability - computer program makes ;the 
selection of a .particular media o*r lesson on the basis of 
availability. • , RATING: 

4. Based on Within-Coarse^ Student Performance - computer pro- 
gram selects a particular media or lesson'on the basis of 
the student's prior test scores, times -to-complete, or 
wi thin-course motivation/interest. t 

- * . • « N *' RATING: ■ . 

5. Based on Pre-Course Student Characteristics - computer pro- 
gram selects a particular media or lesson on the basis of 
-the student's general aptitudes, abilities, attitudes, 

interests or other pre-course data. RATING: — 

6. , Computer SeTections Based on Logical Rules - simple 

"if., then' 1 logicaj rules are used in the selection of media 
er lessons. * RATING: • 



7, Computer Selections Based on Performance Predictions - 

student characteristics and performance are used to predict 
which media or lesson isibest for a given student. 

RATING: . 

Assignment to Alternative Remediation Activities - Providls for 
assignments to available alternate remediation materials and 
media (e.g., CAI reviews, instructor tutorials, etc.).' 

• - RATIH&: 

U Lessons - the computer assigns each student to the best 
available remediation alternative for second and subsequent 
attempts to master lesson objectives. RATING ^ - • .;. , 



70. ?3 J 



ERJC 



2. Lesson Groups - the computer assigns the best available 
remediation alternatives to each student who 4s having 
difficulty mastering the objectives 1n a related group 

. of lessons. . f RATING: , 

. . . jfc « • ... » 
• • • - • - • . v - % 

3. Block or Course Level - the computer assigns the best avail- 
able remediation alternatives to each student who4s*hav1ng 
difficulty mastering block or course level objectives. * 

^ - RATING: . . 

E. Test Selection and Assignment - Different forms of the tests 
can be selected and assigned by the computer. The computer 
can «also determine if a particular trainee should .take an on- 
line (CAT) or an off-line (paper and pencil) version of a test. 

• . * RATJNfc: 



.1. Assign the different forms of a test at random. 

RATING: 



2. Exclude versions that were taken by the trainee onW 
earlier attempt. RATING: 

3. Computer determines whether* to assign on-line or off-line 
< version of test. RATING: . 

F. Assignment to Supplementary Ski \\ Training at Course Entry - 
Provides for assigning those students identified as having 
; deficiencies in critical entry or learning skill areas to 

- • 'supplementary or remedial skill training. Students would be 
assigned in these areas, prior to beginning training or early 
in the training process by specialized materials or instruc- 
tional procedures. * « RATING: 

* 1. gased on Critical Entry Skill Assessment r for example, 
students would be assigned to materials to Improve their 
reading skills or mathematical skills prior to beginning 
training. , RATING:, 
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2 . Based on Deficient Learning Skills Assessment - for exampl e ' 
"students would be assigned to materials to remedy defi- 
ciencies in general study hab|£s and skills, including con- 
centration skills, memorization skills, and test taking 

skills. RATING: . 

* — — — 

G* Additional Resource Management Cons f derations - The system can 
determine which resources are needed by a lesson, if they are 
available, and where they are located. Managed^ resources can 

' * include classroom spaces, lesson materials, media devices, work 
stations, simulators, and other train4ng equipment. Students 
can be assigned in a manner that will even out'the distribution 



V 





of students throughout the cow3cs£ and that will attempt to keep 
as many assignment options a^il^fble for the, student as possible, 

v " '.. RATING: 



1. 



„_3ent assignment to the learning center or classroom 
with the most room (other instructional considerations 
t>e1ng equal). \ * > — - RATING: 



2. 

r 



3. 



Student assignment to blocks, of instruction to achieve 
an opt 1 mat distribution among blocks (other instructional 
considerations being equal). RATING: ^ 



Capability to assign students to resources that are outside 
their, own learning center or classroom (that is, capability 
to share res^cces among several classrooms). * 

RATING: ,. 

"III. ISTUpENT PROGRESS MANAGEMENT - Computer-based capabilities to f 
support the management of a trainee's progress througJ? a self-paced 
training process. These capabilities range from simple scheduling to 
sophisticated progress prediction and management. Both the student and 
instructor know how the, student is progress ing^at all times - on 
schedule, ahead, or behind. 



A. 



A. 



'Scheduling Student Entries into the Course - Notifying $;he 
student squadron or" other authority as to which student can 
start the course each day based on available space in Re- 
starting block or blocks. RATING 



B, Graduation Forecasting - Predicting, m the basis of the stu- 
dent's performance in the course, the "date on which he wttl — 
complete the course, barring subsequent absences. Predicted 

. graduation dates can be shown on the Learning Center roster 

and special reports can list the students predicted to graduate 
* within a specified number of days. Required out-processing 
activities can be scheduled, reports %r the Base Personnel 
office can be prepared, or the System can be directly inter- 
faced to the Personnel Computer System! RATING: 

! 1. Predicted graduation date shown on roster RATING: _ ± 



2. Predicted graduation dates within a specified 
number of days are available by special report 

RATING: 



3. System-generated reports to Base Personnel Office 

RATING: 

4. Direct interface to Base Personnel Computer System % 

RATING: 
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5. System scheduling of out-processing activities 

' RATING: 



C. Monitoring and Management of Student Progress to a Target - 
Establishing a target completion date, for the whole course or 
for individual blocks for each stMdent on, the basis of pre- . 
course characteristics (e.g., ASVAB or Pre-course test scores) 
or performance in the early blocks of*the course. Both the 
student and the instructor will receive feedback on the stu- 
dent's progress relative to his target; Targets can be made 
relative to either average actual course length or the POI 
course length. , RATING: 



1. 



Targets and management printouts addressing the whole 
course. RATING: 



2. Targets arfci management printouts addressing individual 
blocks. . RATING: _ 

3. Use of pre-course data in establishing target dates. 

RATING: 

\ ■ 

4i Use of wi thin-course data in establishing target dates. 

RATINGV 

5. 'Targets are established* relative to average course length. 



RATING: 

6. Targets are established' relative to POI course length. 
y RATING: 



IV. SUPPORT -FOR GUIDANCE AND COUNSELING - Computer-based capabilities 
to- support student guidance and counseling by instructors. and course 
management personnel. These capabilities can be based on identifying 
entry skill deficiencies, deficiencies in wi thin-course performance, or 
both, instructors can quickly determine which students need counseling 
and can retrieve from the computer the information-needed to give mean- 
ingful guidance to the student. - » 

A. Identification of Stflflents Needing Special Attention at Course 
Entry - Provides for the identification, and reporting of stu- 
dents expected to be proficient or deficient with respect to 
mastery of course or training objectives. This identification 
and reporting is initiated at the beginning of the training 
process to focus guidance and counseling functions on those 
students most in netid of ^special training procedures. 

• ' • I •- RATING: 
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Students Flagged on Pre*deter?roipfed Entryyariabfes - for 
example, a list of students measured as proficient of as 
deficient on prttcourse assessments can be printed. 

RATING: 



2. 



Proficient and Deficient Student Scores Printed - besides 
,1 is^ino; students expected to pe proficient or deficient", 
their scores on selected critical entry, skill variables 
(e.g., reading, math, study skills) can also be printed. 

RATING: r 



3. Scores for Al 1 Students on Critical Entry Variables Printed * 
a report which includes the score's of all students on thosje 
variables identified as critical entry skills, flagging * 
those student scores which are deficient.. Allows- extra 
capability to watch students who may be marginal, 

RATflG: j 

4. Prof i'cient and Deficient Student Predicted Completion Times 
Printed - in addition to the foregoing information, the 
predicted training completion times can be printed for those 
♦students expected to be proficient or deficient in mastering 
training objectives. , RATING: 

B. Identification of Marginal Student Performance Wi thin-Course - 
Provides for the identification and reporting of students 
expected tov have* di f f 1 cul ty master 1 ng t ra 1 ni ng objec t i ves . i 

* Identification and reporting are Initiated during training to % 
direct special help to those students most in need of it., | 

# RATING: . 

§■ 1 . . Students Flagged on Pre-detenrtfned Time and Score Variables - 
for example, a list of those students exceeding "predetermined 
training time cut-offs, or scoring below predetermined t 
criterion levels on training objectives, can* Be printed. , 

t RATING: 

2. Times and Scores for Marginal Students Printed *- for " « r 
example, besides listing students whose performance. in 
training is marginal, their actual time and score data can, 
also jae printed. - •» RATING: ^ 

3. Students Identified for Possible ETimination'from Training - 
of students Jisted as marginal , those whose time or - score 
data! are below course-establlslied minimums for acceptable 
performance would be flagged as Rpssible candidates for 
elimination. - r RATING: ■ 



4. Students Identified for Specialized Remedial Training - 
students identified as marginal on the basis of their train- 
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ing tiroes or scores receive specialized remedial training 
assignments {e.g., to special instructor tutorials, r 
remediation sessions out-of-class, or special skill training 
for remedying learning or study skills weaknesses), 

RATING: 



V. GOMPUTER^AIOED 1NSTRUGTI0U (CAI) - Lesson materials are stored in 
the computer, and on request are displayed to a trainee for study; The 
Stored materials. can include text, questions, and^graphics, The trainee 
may interact with the computer "through a keyboard^ by pointing (touch 
panel), or with a light pen {pointing to or drawing on the display 
screen), -Student interaction can range from "turning the pages" on up 
through very sophisticated exchanges - for example^ In simulated trouble- 
shooting, var^o^s information can be "requested fronrthe computer , and 
various solutions can be tested. - ) ' 

A.' CAI A&pli' cations - Ther,e are five different ways in which CAI 
can be used tn Air Force 'training: drill and practice - 
practice with ^correction and* guidance of basic skills and 
knowledges learned via other media; tutorial - the use of CAI 
for particular whole lessons which are particularly difficult; 
simulation -""use of CAI to simulate equipment or processes and 
to teach students to use the equipment or follow the process; 
review* and remediation - use of CAI to quickly review the con- 
tent of a block prior to a block test or to Yemediate over 
failed objectives following a bjosk test failure; and study 
skills - use of CAI to teach specific study skills to students 
who are deficient in these^areas. . RATING: 



V, 



lu Drill and Practice CAI in your training trea> 

? ? R|TIH6: _ 

2. Tutorial XAI in your, training area. GATING: _^ 

3. Simulation CAI in your training area. RATING: _ 

4. CAI for Block Review prior to the block test. 

X # • » . •,. RATING: . 

5. CAI for Block Remediation after a failure . RA TING: 

6. CAI for teaching study skills, RATINGr 

B. CAI Capabilities - Regardless of the particular, applications, 
i the sophi s ti cati on of the CAI capability can vary from lean t# 
rich. The most basic CAI system would present text, and allow 
input to multifrte-choice questions via a typewriter keyboard. 
The CAI programs would contain the capabil ity to adapt to the* 
student's responses but they would not access other information 
about the student stored in the computer. A richer CAI system 



will support graphics tHiie_ drawings) as we31 as text, allow 
students to entefvconstructed-response (open-ended) answers, to 
respond by touching the screen (using a touch-sensitive 
surface), or with a light-sensitive pointer tUght pen), | 
control a supplementary slide or film-strip presentation, 
control an audio tape, and retrieve other information about 
the student from the data base to allow' more adaptive 
instruction. . ' • v " 

1. Basic CAI * presenting text, and allowing input to multiple 
choice questions from a keyboard, • Has, the capability to 
adapt to the answers, but does not access other information 
about the student . RATING : , 



2. Adding grapfiTcs" capaBt TTty~E6~Tfie "Basfc CAI. 

3. Adding constructed response test items (Engl 
answers) to the basic tAI. 

4. Adding touch-sensitive or light. pen capabili 



RATING: 
ish 

RATING: , 
ty. 

RATING: 



S. Adding audio tape control jto basic CAI - canl 
present voice accompaniment during selected I 
parts of the lesson. \ IRATING: 

Adding computer control over a slide or filmsltrip 
projector - can present, supplementary visual material' 
• to accompany the CAI display in designated parts of. 

the lesson. RATING: 

* '% • 

7. Providing additional student information f rom *the 

computer data base - for example, ASVAB or AFQT scores, 
reading ability scores, or performance on an earlier 
lesson t- to increase the sophistication of adapting 
the CAI orooram to the student* s abilities and interests. 
• ' RATING: - 



On-line Learning Process or Specialized Skill Training - Pre- 
sentation of specialized training In. a highly interactive, 
computer-assisted fcrmat to either students or instructors. In, 
the case of students, the on-line materials would be designed 
for learning skill training (e.g., study habits and skills, 
test-taking skills, memorization skills) for those students 
identified as needing this training. In the case of instruc- 
tors, the on-line materials would be designed fco strain 
diagnostic, tutorial, or other counseling skills for those 
instructors who volunteer or are selected for this training. V ( 



RATING: 
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1, On-line Student Skill Training » for example, for students 
wenti f led as > deficient J u critical entry skills or as 
haying marginal witf)1n-course performance, assignment to 
specialized on-line materials would be provided. 
• RATING: 

2. On-line Instructor Skill Training - 'for examp%, for in- 
structors who volunteer or are selected for §pfecia|ize# 

i skill/role training, assignment to specialized oi^ne 
. ; training materials and sequences Would be provided. 

JfcTIHSt 





I. GUT DATA BASE AND CAP MATERIALS PRODUCTION AN0 : MAINTENANCE :- To 
erform any training functions, th€v computer must be informed about the 
e&iaane* social securfty nrter, course, etc.), tne course, 
(tests,- lessons, resources^ allowable sequences of lessons, 
, . Sstudent performance 'flessofis completed, test scopes, block ; 
ompletion times, etc.). x The CMI Data Base refers to entry/ storage, 
nd maintenance of this information. f •:' 

^ > ' • - " -i-S; .:• ' . 

in the case , of Computer Assisted Instruction (CAI), all of the 
esson Information (text, questions, graphics, branching Instructions, > 
tc.) must be stored In the computer. CAI Materials Production and | 
iaintenjatfce refers to entry-storage and maintenance of this information. 

A, CMI Course Definition Data Base Editing - Used by course person- 
nel to enter the information that describes the course structure 
the computer. Tills editing should 'be easy to accomplish and 
ksy to modify. It' can be. an off-line card based system (c6urse 
irsonnel describe the course structure On paper, and Jater -C 
receive a printout listing what has been entered in tor the data 
oase), or can run on-line (course personnel at a computer 
terminal enter, display, and change the Information directly}. 
To facilitate use, the on-line system can use graphics to diss 
play the course structure (the allowable paths through the | 
1 esson Hnaterials).. '** * ; « 

jr 

Is. leaking the system* on-line, 'rather than fhe off-line 
. card based approach. , RATING: 

2. Using graphics to display the course structure, rather 
than relying completely on words;, RATING: 



B. Aujteaaatic^ Ity. 
to check automatically if til required data Sase records ato 
^describe the course are present* and consistent. When a new 
course or course version is implemented, or an existing one is 
changed, all of the lessons, tests, and blocks must be in the 
data base. This capability will automatically check for 
completeness. -RATING: 
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S^^^f^J^^^SS^Sl Qn-Line Tests - If the System, includes- 
the capability for on-line testing, a means must be provided to 
enter the test questions and their alternatives 'into the \ . 
computer system 1 s data t>ase - a Test Item Editor, The simplest 
form of .the Editor would allow the 'author to type in text. Add- 
itional feature* jNtouTd format the test ■ question for the author, 
■allow *he author to*decide whether Or not to randomize the 
presentation order 'of., test items and of al ternatives within - 
test items, control the number of attempts the student could 
make, on each item, and enter feedback messages" to follow 
correcf and incorrect answers. * RATING: 



1. 



2. 



Adding automatic formatting for the entry of test 1 
questions; — — — r -.,... • RATING: 

' ... r 

Adding 'author control over whether order* of test . 
items and alternatives will be randomised. 

RATING: 



3. Adding author control over the* number of attempts 
allowed on a question. RATING: 

* . . . ■ ♦ . 

4. Adding feedback messages for correct and Incorrect 
answers, at the author's option. • RATING: ;. • 

CAI 'AwtKoring Editor - If the System includes CAI, there must 
be a means of producing the CAI material s< or programs. The^ 
traditional approach is to have CAI authors learn an authoring 
language,, and use ft iri f writing their lessons. . jtltiernrttv* 
is to have CAI lessons written by an author/ programmer team. 
These approaches require practice and experience. Another 
alternative is to provide a CAI Authoring Editor which structures 
the 1 author's task for him, provides standardized formats for 
text frames and questions, and eliminates t$te need for any pro- 
gramming abi 1 fty . The program format would not be rigid and 
alternative presentation approaches can be provided by the 
Editor. Instructions for using the editor can be in a manual 
and in addition, instructions can be displayed in the Editor 
program itself through a HELP request. The HELP information 
can refer just to the operation of the Editor or can also 
provide guidelines for good Instructional practices.. 



V -v 



1. Providing a CAI Authoring Editor, rather than 
CAT' author 

with a 1, programmer to construct l$ssorjs.- 

2* Providing, in the CAI Authoring Editor, automatic 
formatting for text frames and for questions. 

.V RATING: 



uirtng 




■1 
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3. . Providing, 1n the Editor, automatic structuring of 

the sequence of frames and questions 1n tte CAI 
presentation. * * * * RATING; 

4. 'Providing capability .to copy text frames, questions, 

* ;and graphics from other CAI lessons. RATING: 

5r /Providing a HELP request to display instructions for . 

* u$1-ng the Editor instead of relying entirely on an V 
Instruction manual . - RATING: ^ 

6. s Adding guidelines fpr good instructional practices . 

as part of a HELP request at the terminal, RATING: ; 

Usangf^Stnw " 

Matter Ifrts - If Test Item and CAI Authoring Editors are avail- 
ably, it may be useful te use them for producing tes^s and 
instructional materials which are Intended for use off-line, 
e.g., regular programmed texts. The Editors can be used to 
write, review, and revise "the tests and materials and the final 
product can be printed out' ready to be reproduced. If the 
number of copies required is not too great* multiple copies, can 
be printed to avoid* the need for reproduction. Finally, small- 
group tryouts could be conducted on-line |1 ike CAI) to provide 
more detailed information on students' reactions' to the * 
materials. f* , RATING: 

1. Using the Test Item Editor to construct and produce 
• off-line tests. • RATING: 



2. -Using, the CAI Authoring Editor to construct and 
produce bff^line lesson materials. RATIN^j 

3. Using Aitors. for review and revHionof tests and 
lesson materials. \ *. • ^ RATING: 

4. Printing out k master copy, rea# for reproduction, of 
tests and materials which have been written using the-, 
Editors.*! • ' " RATING: 



ERIC 



5. Printing out multiple copies of. tests and materials, 

in order to bypass reproduction for small quantities. > 
■ ■ - RATING: *_ 

. - t - - - , . ^ fl*. . - - - v . , ^ . 

. » * .,. * * » - 

6* Providing capability for conducting on-line small . . 

group tryouts of materials and tests. RATING: >• 

F. Use of (Jn-Ltne Editors for Production and Maintenance of Course 
i ^cuiiiiht^ - An fcdi tor like those described -for Test Items and 
CAI authoring could be used for producing course documents such 
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V 



the *R0>, course charts * . l«sstfii^il£M^ s > ttFfe Tt»e documents 
would .then be avail able at any. time; through* tSP tdi^tors- for 
review or revision* and o«#ies coul d be pri tfted .out' as needed*. 




til*" INfORMATIOW RETRIEVAL AND REPORTS* * iC<^COri^uW-b«seji training * 
system", Targe^ amounts^ of information ajiout- trainees,' courses, course 
material's', tests and**testi terns, nnd o|her |)arjts of tne system are' ! C> 
stored, and- a*rje available fpr - pt^rfjpct^ *^) | 'wiiiJljjte^a*t ly »• 
jt'morttfcly. " v > ^ ajwjyses ..anS r^'r^J.;^esb .fufRlons^are intended- , . 
-principally "for instructors,, managers, and administratprs*. .they can be 
valuable to instructors Who must .provide management and guidance to' * 
sttittents. ' They al so '.can be valuable, in providing\quai fty control over/ 
sTJodents and materials, in improving the management and administration 
of, training, , and ifir^luating and improving all aspects of development 
and impl ementatiO;n\ of :*trai hi pg materials and programs. 

; • V ' •*" •- %" , ' • • <v ... • ' 

■ •* - ' •» , » * - * ' * 

A* . Reports f or I nstructor s rvtarge amounts 'of data can Be collected 
and made avaTTaBle to Instructors, but the .reports, should be ; 

* ' carefully structured to meet the needs of the instructor^.*. The 
v reports may include: classroom rosters,, summaries for specific 

• . v parts of, the course, inuivfclus 1 history reports, and suWaries 

for tty. classroom. V * * . RVflNG: ' t 

**> . * * 

v 1. learning Center or Classroom Rosters - Rosters can be 

pi ayed . or pri nted ~o show many di f f erent kinds of inf»-»rna- 
* tion about each, of the students in a classroom or learning 
. , ' . ' center. *The basic ^formation would inelu^^ security 
, ; # „ numbers , names, carrel (or .position) nufiberi, absences, 

* , current "block, -and rate^f progress . ,\» RATING: ^ 

• * • . . 

, * 2. Individual Block Progress Reports - The computer system 
can allow review of data by instructor^_^ - >tf|fervjsQrs, 
counselors, etc., vta local. .terminals. Progress reports 
. , •» can provide data for in-course counselling and can become 
•»*• . economical, replacements for .some of the current" ImanUa 1 1 y 
} \ * prepared vforms which accompany student records throughout \ 

the course and,* in some cases, beyond graduation. \ An 
Individual Bljf&k Progress Report will summarize student 
progress, times and scores, and* other pertinent information. 
' * < . ; x ' * RATING:- 





Information included in the- block progress repprts can w 

include some or all of tfce following: * , .'.V 
' • •.. - • * . " • • * * is 

a.) lessons completed by the student i ' * ••, RATING: ^ . \ ; , 

b ) ^numbers of attempts » an^ scores on each^ 
* attempt, on' each iessojpr* - > » , 

\" ' > l »' «\ '"V.-' . + 




RATjlNG,: 



v v ... ' » 



S SO* 



c) objectives failed storing, the block; RATING: 

d) time spent on each lesson and on the 

block* > V • . RATING: 

e> amount of time student is ahead or behind 

\ schedule; - ' , RATING: 

*f) amount of. homework student- has accomplished. 

.,' : 4 • , , ... ' * RATING: 



Individual Student History Reports - The system can provide 
*' a -Targe amount of additional; data* about an individual c . 
student. This can jnclude tyre student's sa>res on precourse 
aptitude and interest measures, scores on critical entry 
skill tests, time spent In block remediation following a 
bltfck failure," and absence times* and reasons for absence. • 

v ftf v.. i'\ RATING:-, 

I nf ormation i n the * I ndi vi «fcl Student. History Report c£n 
include -some or* all of the .following: 

a) attitude and interest information; RATING: r •„■ . 

b) scores on critical entry^sMUs* RATING: ; . 

c) time in block remediation following a bloc* • * 
-failure;. ' '. ' RATING: ■ 

d) lesson objectives that were failed more than - 

• twice; ' • - , ^ v mim: 

*" -v^ , A t , • ■ t 

e) time* absent in each block, and reasons for 
absences.' v V ,; ' . . RATING: 

Additional Reports on the Student - 

" ■ " ' * - • • ' * ? 

a) Absence Report, to replace the manual" absence 
report;- „ RATING*. 



b> Homework Summary Report, Xyl list lessons accomplished 
* * as homework and to summamze homework time expendeo/; 
* ' RATING: 



,c) On-line display of student- data - the capability to 

call up and display* any of tfhe- avail able student 
* information. . ... ! RATING: _ 



Training Management Reports>. Reports- such as these are- intended 
for use by managers' (Shift Supervisors '-and above) in monitorirjg * 
training activities.' Potential reports include number of stu- 
dents awaiting training and Jiqw long they have been waiting, y 
number of trainees in each block oV each classroom on each base, 
the distribution of .completion times 'for bldck$ and^the course 
itself .differences in trainee performance between parallel 
classrooms or bases i the extent to which instruction^ resources 
such as carrels', equipment, and training devices are -being 
^utilized, and summaries, by -block' or other Unit, of trainee per- 
formance. * . * ; • 

1. Report on Students Awaiting Training; RATING:. _ 

2* Report of number of students currently in each block, , Y • 
* classrooto,, or on each base; \ . RATING: 



3. The distribution of completion times for individual . . 
* blocks;..* * « •; 'RATINGS * 

C The distribution of 'course completion times; 

* - . ^ • ' RATING: 

5. Differences in student performance across parallel i 
learning centers , classrooms b> bases; . v 'RATING: $t 

6. Reports of instructional resource (faciliftes, 

training davlees^ etc.) utiliaa^en; RATING: 

7 ? Report of uti 1 i zation and performance of. _ / 
. ' instructors. * , ' ftATlNG: 

8. Summary Reports of student* completion times liftd : 
final grades* by^Mocfcg ... RATING: » 

Course Eval uatiort Summary - Summarizes student performance on 
indivldu|il lessons withi?i*blo|ks*and or! the block Itself. t *tt is 
intended* for use 1 n smal 1 r group tryouts of new mater i al s and 
monitoring student performance on established materials. The 
Course* Eval wStion Summary can eontai n a variety of features , 
listed below. * * " RATING: -\ 

K fc Firs t-attempt*-! esson anS block test scores. RAJlVfef 1 ■ 

2'. 'Final .lesson and. block test scores.! . RATIN6r*__ 

3. 'Mean and stantiard^eviation'of first-attempt lesson 

and b'loc^L test scores; •; < * . \ 4 -*4UAG: J 



4. Mean and standard deviation of final lesion and., 

' block test scares* * : - ; RATING: _ 

*5. Mean and standard deviation of first-attest lesson 
and block: times. . * RATING: . 

6. Mean and /standard deviation of lesson and block 
completion times. • * RATING: _ 

7. Separate score and time data for alternative modules 
within lessons** ' , . . RATING: _ 

8. Score data broken out by objective within lessons. 

. . ' * , ' RATING: 

* * » 

9. . First-attempt "tesson failure fates. , RATING: 
VO. First-attempt objective failure rates. RATING; 

11 . Correlations *nowi^ % thtf relationship between 
predicted and actual lesson and block* times 
and scores; v - RATING; _ 

12. Summarized !data on lessons, modules and objectives . 
* with first-attempt failure rates above a certain 

/ percentage,* ; RATING: „ 

Tes t Item Eval uation Report -'Provides detailed information on 
the characteristics, of bl<ock and lesson tests, rt could con- 
tain a variety of features,, listed below.*' RATING: ^ 

1. The number (or percentage) of- tral nees Select 1ng each 

alternative answer to test questions. RATING: ^_ 

H. Percentage of trainees answering each questjdn 

correctly. . RATING: 

3. Average test score of trainees selecting each 

v .alternative on ea£h question, RATING: , _ 

• * » ** * 

4. Flagging .test questions missed by over* 70 percent 

jv j*r by none of the trainees. 1 • RATING: 

* / * 

5. I tern- remainder correlations for each question. 

I RATING: ' 

- » > * , *• 

6. Means and standard deviations, of scores for 'each, 
objective and for each full tSst. ' / RATING: 
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71 Distribution of scores on the full test. - RATING: J , 

8. Alpha relfabl 1 1 ty 'coefficient for each objjectl ve 
. atuffor the fuirtestu* . * * RATING: . 

.9% Cir1 terl on- referenced reliability coefficients .for ; 

each objective and for- the full test, RATINGf 

■ : ■ '■■ • ' , . \ \ • T\ •. ' 

E. CAI Evaluation Data - Since^CAI Involves frequent interaction 
wi th tlie student, very detai led eval uation -data are available 
from CAI programs';- The most' frequent use of this information 
would be durin^fonnative evaluation' (small -group tryouts) of 
the programs. wvariety'tff capabilities are listed below. 

1 . Percentage of s tudents selecting, each* al ternatl ve i n 



each question frame* * '"RATING! 

> • • \ 

2. *Perce*age of students selecting' each alternative 
fn eacn .question « frame on their tfirst try, thefK 
second try, etc. * t RATING: 

3. The means and standard deviations Of the time tb' 
respond to question frames and the time to read 

; text frames. „ * % . RATIMB: 

4. Lists'" of unanticipated responses to constructed 
response -questions. - *' \ RATING: 

5. Lists' of open-ended student comments about tyie.CAI 
program. * . - RATING: 

6. Detailed records* of each student's path through a t 
CAI program. • •? RATING: 



■ • 



7. a<!raifer/stati sties on major units of . the program,/ ; 
•. suclvas time to. complete and cumulative score,** 

RATING: 

F. Courseware Devejojment Management Reports- -« When* Edi tors % are used 

for developing on-line test§,CAi materials, or tests and , 
_ materials to bejused off-Tine, the System can capture information * 
" -about authors' work on the Edi tors, and the status of the * 
developmentveffoft. This information could 'then be s ummarl zed 
tn management reports fdrmoni tori ng ma teri al s produc ti on*, H « 
review, revision, and implefientatlon. The report* could include ^ 
information such as. the person responsible for developing,; 
previewing,. or revising a lesson, how long they have been wr*- „ 
ing on It, how much ot the work has 'feeeh completed, and expected 
( completion date. • \ ' ■ RATING: . ^ ' 

-• ., - .» - • - " . • . 

« ,* • % • _ * » , 
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Performance Forecast Reports --provides* f 1 el d , OJT, or technical 
training personnel with tralnfttg 'Information which describes * 
current s tudenf performance 1 evel r or predj ets how 1 ndi vi dual 
students win Jeff orm in specific training areas. • 

. . • . RATING: _ 

1* Pre-Service Education and/or Vocational. Training - available, 

Jndlvi-dtial 4ata relevant to prior education or training. 
> V \ - RATING: . 

2 . Pre-$ervice\Occupa tj onal / Job Experience. - data 

which reflects prior relevant job experience. v 

r ' "RATING: . 

3. Enl i s'tment/Cl ass 1 f i cation Data - data- available in AF 
personnel records (e.g. , ASVAB scores* AFOQT or. AFQT _ 

' scores). RATING: 

4 . . In-Service Training Performance Data - for exampl e , for. 
FTD or OJT managers^ a report of individual student' * 
technical training or in-service training performance , * , 
data^fe.g., scores, times^.to^complete). RATING: _ . 



5. 



AF" Special ty-Related Deficiencies - *f or .example, student, 
data on deficiencies in skil>svexper1ences\ or abilities 
• relevant" to that particular i ndTVidual ' s .special ty . 

» A RATING: 



6. Student$' Predicted Adaptabilitj;. Performance and 

Pr ogression in Operational Field Assignments - for example, 
jdentfc' in*-service and pre-service data could -be used to 
repdrts of predicted field assignment adaptability, 



studenU 

provide . - r ~. „ , . r 

performance and- progression. RATING: * 



"fo/m 

Special Purpose Reports - No matter how extensive, no set of 
standard repots wl 1 1 always- be enough to answer all management 
and evaluation questions. It is useful., therefore, to have an • 
open-ended information retrieval capability that can be used to 
answer unexpected questions, about student and System performance. 
A minimal capability would permit the user to define the 
Variables which he wishes to look at, and the types of students 
and fime periods from which the data are- to be drawn. The out- 
put would consist of the number of cases found, and the mean,- 
standard deviation, and range for each included variable. Such 
a program could be rim ; interactively, in a. minute or" so, and the 
results displayed on .'the user's scjreen or it could be run^as a 
background job, in a half*hour 'or so, and the results printed 
• ou t . . * . ' . 5 



i 
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ll Utility of basic Data Retrieval Program with owl/ 
i; background capability. ) RATING: 

2. Utility of adttMtg interactive (as well as back- 
ground) capability. * >A RATING: 



3. The basic data retrieval program>c$uld also be inter- 
faced to one of the standard packages o,f statistical *\ 
- programs "(for f^ample, S?SS, or BMD). This would permit. ~ 
extensive analyfis of student data and the capability ' * 
to . answer "lust jfcout any question that could be asked ^ 
about the dataSrollected in the s*/stem. Some training 
V is, however ^required to use these statistical packages. 

r RATING:- 



■» 
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• £ ; APPENDIX B / * " ^ 

INTODOUCTION TO SURVEY FORM AHO. EXPLANATION Of RATING" SCALE 
• . IPEWIONAL COMPUTER-BASED TRAINING SYSTitf RIQUIipifikfS \. ; 

A highly sophisticated, prototype computer-based -training system has 
teen developed and Impfemented at'Lowry AFB; ^TWs Adian£ed Instructional 
System, or AIS, represents,;^? stated fche-art in^tnlnsiructional 
an^compujer technology, anjl has Contributed to training time savings ' 
of. 30% or more in a varfefy of technical* training areas. As a prototype 
system, however, the AJS provides functions *nd capabilities ifeat may 

^;J® CfiS ? ar y the operational %a1ning environment^ Totalize 
the full potential of computer-based systems for Air Force training, we 
Want to firnd out what function*3or capabilities will nave the highest .# 
payoff for operational trainiqM " ' . • 

You have* been*selec ted to participate in ihis survey to determine 
what fdnctigns are most valuable in your training area. Your assistance 
in carefdlly analyzing the value of various computer-based functions is 
extremely important. Your ratings, along with the ratings of the other * 
participants, will be used in defining a low-cost cej^uter-fflanaged* and V* 
computer-assisted instruction |pw/CA,I) system for Air force training. $ 

The following pages describe the many functions that a computer 
could perform in a training system. Please read the description of a 
function, and then carefully consider what that? function could accomplish 
in your nartioular area of trailing. The question you are to answer, 
for each function, is: 

"HOW MUCH IS THIS FUNCTION WORTH TO MY AREA OF AIR FORCE TRAINING?" 
Payoff potential can come from any of the following: 

Reduced training time, students get through faster;... V 

\ Reduced administrative load on instructors; 

. ... Better use of training resources; 

Q Improved quality controT over students or materials; 

Improved management/administration of training; and 

0 Improved. materials development process^ resulting in better in- 
structional materials and reduced development time. 

On the following pages, there are eight brief .general descriptions 
of various areas in whic% the 'Computer could provide training support, 

Jjlnder each of the general area descriptions, specific areas of 



computer-based training functions "are described. You are asked to con- 
sider how much each of these might be worth to Air Force training in 
your 1 particular area, and to assign a rating of: ' 

.> . ' v- ■ " : • • ' ' *. ; - :: • . 1 * '* • • 

1 = Very high payoff potential; 

,,2 * High payoff potential ; , 

3 * Moderate payoff .potential; 

4 a SmaTJ payoff -potential ; * 

5t = No payoff potential; and • • v 

. X = Ho opinion - this fupction's effects would be outside; ray^geea, 
and I cannot estimate Its payoff potential. . : 

If* there are functions or- features that should be included in a 
computer-based training system for your area of Air Force training, and 
that are not listed, please describe these on the last pagefof this 
packet. 

If you are involved in resident training, then assume thai your area 
of training would be self-paced before the computer-based train ihg system 
is introduced; This is important", because many of the computer functions 
could be quite valuable in self-paced training but of little vaJue in 
conventional lockstep instruction. Remember, as you assign ^tlngs.^that 
you are to consider thi payoff of each function in self-paced training. 

. t - If you have any questions , pi ease ask the represe^ati ve conducting ^ 
this survey. \ 
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LISTING OF CAI/CMI FUNCTIONS, IN THE ORDER U$ft> (JN THE SURVtTfORMS^ WITH MEAN RAT tngs pphm DcctncwT 
TECHNICAL TRAINING, AND ESTIMATED SYSTEM IMPACTS (TILE REQUIREMENTS AND ^^Jtf^H^Jf?^ 
WHICH ARE RECOMMENDED FOR OMISSION f ROM THE SYStS n^am^^ki^SP 

' fSf fsf iSlfSiJ ? IS* f BJ * n ^ The W ^ appeared only on the 4*s trustor survey 
s^rVtsor ^ 011 ^ . Sttperv1sor f 0 ™^* 1 *** -W«k Items appeared on both, tnstructo? 




####### 
IA 




(S) IB 




i - student testing and .evaluation * ####### 
test scoring and 'feedback » 

' righf^jrong with percent correct 
correct score for guess i ng 
weight 'questions differlnti ally 
pass certain percent of objectives 
pass specific objectives 
pass specific questions * * 
score performance checklists 
instructor can certify lesson passt 

(note: ia7 and ia8, rated low but needed) 

additional on-line test capabilities *v' 

on-line, random question orders 
on-line, Random orders of alternatives 
* on-line, retest over failed objectives 
on-line, co!ct«jcted responses 
on-line, adaptive within test 
on-4ine, adaptive on course performance 
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/ 
/ 
/ 
/ 
/ 
/ 
/ 
/ 



2*20 
3.20 
2.60 
2.4$ 
2.2S 
3.3Q 
3.Q6 
3.34 



NEEDED. 

f 
K,R,S 

K,S 
K,S 
K,S 
K»S 
K,S 
K,S 
K,S 
Kf S 



TOO MSEC/ \ 

YransactiKjn 



/ £.26 I.l^S.T 



/ 2.30 
/ 2.31 

I im 

I 2.85 
/ 2*91 



I,K,S,T 

'"X- jfJC 

CT 

CB»CO-,CT 
CB,CO,CT 



5 MSEC/SEC/ 
STUDENT 



APPENDIX C (CONTINUED) 



FUNCTION 



I 1 ! 



IC PREASSESSftEfr DATA CAN USED 

IC1 RELEVANT AF PERSONNEL DATA CAN BE USED 

IC2 PRE-COURSE KNOWLEDGE TESTS 

) IC3 LEARNING PROCESS SKILLS 

HI) IC4 COURSE-SPECIFIf ENTRY SKILLS „ " # 

ID ATTITODES AUO INTERESTS 

) ID1 PRE AND POST ATTITUDES 

;•)' 102 WITHIN COURSE, ATTITUDES AND INTERESTS 

####### .II - STUDENT J&SIGNMfNT ####### 

' IIA v *ASSIGN.TO ALTERNATE COURSE VERSIONS^ 

TIB DETERMINE' QRDER OF* ASSIGNMENTS ; 

g (I) IIBT ORDER BASED ON LESSONS COMPLETED 

(1) IIB2 ORDER BASED ON EARLIER PERFORMANCE 

. (I) I IB3 ""ORDER BASED' ON RESOURCE AVAILABILITY > r 

(Ip IIB4 ORdER BASED ON TEAM CONSIDERATIONS 

DROP (I) JIB§ ORDER SELECTED BY INSTRUCTORS 

" ' IIC ASSItar m ALTERNATE LESSON TREATMENTS 
• DROP (I) « IIC1 I NSTRUCTOR ASSIGNS ALTERNATES- 
DROP (I) IIC2 STUDENT SELECTS DESIRED ALTERNATES 

(I) IIC3 RESOURCES DICTATE ALTERNATE ASSIGNMENT 



14 



# 



FILES • 

/ x VALUt NEED® CPU TIME 



/ 2.57 

/ 2.53 S BATCH 

/ 2.3$ K,S 

/ 2. $9 K,S 

/ 2.40 K,S,V ' 

7 2.66 K,S 
/ 2.78 K,S 
/ 2.82 K,S- 

/ 2,80 f*H,S * * 

/ 2.56 H 
/ 2.56 HyS 

/ 2,55 H,S 4 MSEC/ 

-•V* TRANSACTION 
/ 2.50 F,R > 4 MSEC/ 

TRANSACTION 
/ 2.74 H 3 L»S 1 MSEC/ 

TRANSACTION 
/« 2.93 . D,L,M, 3 MSEC/ 

. X^»S * TRANSACTION 
/ 2.70, - ~ • • -• 
/ 2.79 L»M,N;S 
/ 3.71 D,L,M,N,S* 
/ 2,89 F 3HSEC/. 

TRANSACTION 



APPENDIX £ (CONTINUED) 



FUNCTION 



4- 



FILES « 
/ VALUE NEEDED CPU TIME 



(I) 

(!) 

(I) 
(S) 

(I) 
d) 
(i) 



(i) 
(i) 

(i) 
(i) 

(i) 

(i). 
(i) 



IIC4 IN-COURSE PERFORMANCE *DETERMI NES ASSIGNMENT 

IIC5 ^RE-COURSE MEASURE? DETERMirfc ASSIGNMENT 
IIC6 HEURISTIC "RULES FOR ALIGNMENT SELECTION 

.(NOTE; flC6 IS LOW VALUE » BUT USEFUL) 
IIC7 PERFORMANCE .PREDICTION DETERM I NES*ASS I GNMENT 



I ID ASSIGN TO ALTERNATE REMEDIATIONS 

HOT ASSIGN- TO ALTERNATE LESSON REMEDIATION - 

IID2, ASSIGN TO ALTERNATE LESSON GROUPS REMEDIATION 

t ■ 

IID3 ASSIGN TO BLOCK OR COURSE REMEDIATIONS 

I IE .TESTS SELECTED AND ASSIGNED 
IIE1 TEST ASSIGNMENT^ RANDOM 

IIE2 TEST ASSIGNMENT. EXCLUDES PREVIOUSLY TAKEN TESTS 
IIE3 ASSIGNMENT TO ON-LINE OR OFF-LINE, TESTS * 

1,1 F ASSIGNMENT TO SUPPLEMENTARY SKILL TRAINING ON ENTRY ' 
I IF! ASSIGN SUPPLEMENTARY TRAINING BASED ON* ENTRY SKILLS 
IIF2 ASSIGN SUPPLEMENTARY TRAINING BASED ON LEARNING SKILLS 

IIG RESOURCE MANAGEMENT BY COMPUTER • 

IIG1 RESOURCES: . ASSIGN TO LEARNING CENTERS * 

IIG2 RESOURCES: ASSIGN TO OPTIMIZE ACROSS BLOCKS 

IIG3 RESOURCES: MANAGE OUTSIDE LEARNING CENTER 



/ 2.75 H,L,S 



/ 2.84 
/ 3.38 

/ 2*. 95, 

/ 2.61 
/ 2.38 

/ 2.45 

/ 2.53 

/ 2.86 
/ 2.86 
/ 2.52 
/ 2.98 

I 2.28 
/ 2.10 
/ 2.16 

/ 2.48 
/ 2.74 
X2.98 
/ 2,88 



h.l.s; 

«»L,S 

HyLfS 



L,N 

D»L,M t 
N,S 

D.UM, 
N,/ 

H,L,S 
H.L.S 
H.L.S 
H-.L.S . 

H,K,5 
H,*,S 
H»K*S* 

F,P»S ' 
P,S 

F,S 



3, MSEC/ 
TRANSACTION 



TO MSEC/ 
TRANSACTION 
TO MSEC/ 
TRANSACTION 
10 MSEC/ 
TRANSACTION 



1 MSEC 
1 MSEC 
3 MSEC 



f 



APPENDIX. C (COMTINJiED) 



FUNCTION 



TO 



III - STUDENT PROGRESS MANAGEMENT ###### 
IIJA SCHEDULE STUDENT £NTRIES TO COURSE 

II IB GRADUATION FORECASTING 
(I), IIJBl PREDICTED GRADUATION DATES ON ROSTEI 

(J) HIB2 GRADUATES IN NEXT X* DAYS, LISTED Of/ REPORT 

. IIIB3 GRADUATION REPORTS, Ft>R CBP0 
I I I B4 DIRECT INTERFACE TO PERSONNEL SYSTfM 

«♦ j 

' IIIB5 GRADUATES: SCHEDULE OUT-PROCESSIN( 

» MANAGE STUDENT Td TARGET 

(I) I I IC1 MANAGEMENT TARGETS PRINTED FOR WHOLE BOURSE 

• <I) I I IC2 MANAGEMENT TARGETS PRINTED FOR INDIVIDUAL BLOCKS 

' (I) IIIC3 MANAGEMENT TARGETS BASED ON PRE-COURSE DATA 

(I) IIIC4 MANAGEMENT TARGETS, FROM WITHIN- COURSE DATA 

(I)' IIIC5 MANAGEMENT TARGETS RELATIVE TO AVERAGE COURSE LENGTH 

DROP (I) IIIC6 MANAGEMENT TARGETS, RELATIVE TO.POI LENGTHS 



00 98 



ERIC 



-FILES 

/ VALUE NEEDED CPU TIME 



/ 2.39 P,S 
t 2.47 ' 

/ 2.74 C,P,S,W 3 MSEWEC/ 

OCCURRENCE 

/. 2.64 C,P,S,W 3 MSEC/SEC/ 

OCCURRENCE 

/ 2.62 C,P,S,*W gATCH 

/ 2.51 (NEEDS DEFINITION FROM 
AIR FORCE) 

/ 2.58 E,S ( BATCH 

/ 2.83 H 

/ .2.83 C,H,P,S 

/ 2.89 C,H,P,S 

/ 1.02 C,H,5,W 

/ 2.69 C,H,S,W 

/ 2*9? C,H,S,W. 

/ 2.97 C,H,S,W 



* 



00 

r 



FUNCTION 



'^APPENDIX C (CONTINUED) 



####### IV - SUPPORT FOR GUIDANCE AND COUNSELING ####### 
IVA IDENTIFY STUDENTS NEEDING SPECIAL ATTENTION 

. r 

(I) IVA1 FLAGGED ON ENTRY VARIABLES 

(I) IVA2 PROFICIENCY AND DEFICIENCY SCORES PRINTED ' * 

(I) IVA3 SCORES ON CRITICAL\ ENTRY VARIABLES PRINTED 

(I) IVA4 PREDICTED COMPLETION TIMES, PROFICIENCY/DEFICIENCY 

*IVB IDENTIFY MARGINAL PERFORMANCE IN COURSE 

« » 

(I) IVB1 FLAGGED ON TIME AND SCORE VARIABLES ■ 

(I) IVB2 MARGINAL TIMES AND SCORES PRINTED OUT 

(I) ( IVB3 FLAGGED FOR ELIMINATION FROM TRAINING ' 

(I) IVB4 FLAGGED FdR SPECIAL REMEDIAL TRAINING 



FILtS • 
/ VALUE NEEDED^XPU TIME 





2.30 


C.E.P, 






s»v 


/ 


2.45 


C,E,P, 






s,v 


/. 


2.38 


C£,P, 


7 




s,v 


2.49 


C,E,P, 

W w V 






s,v 


./ 


2.64 


C,E,P, 






c 


/ 


2.32 


C,E,P, 




2.55* 


*S,W 


/ 


C.E.f?, 




2.50 


s,w- 


/ 


C,E,P, 






s,w 


/ 


2.26 


C,E,P,' 






s»w 


/ 


2.30 


C,E,P, 


t 







5 MSEC/STUDENT 
CONSIDERED 



J 



too 
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APPENDIX C (CONTINUED) 



FUNCTI(fo 



* ... FILES 
/ VAtte NEEDED CPU TIME * 




####### V - COMPUTER-AIDED INSTRUCTION (CAI) ####### 
VA. CAI APPLICATIONS, TRAINING. WUBSTING 

. N- 

(I) VAl CAI DRILL AND PRACTICE 
H) VA2 CAI TUTORIAL 

* 

(I)' VA3 CAI SIMULATION (LIMITED CAPABILITY) 
(I) VA4 CAI FOBf BLOCK "REVIEW t 
(I) VA5 CAI FOR BLOCK REMEDIATION 
(I) VA6 CAI FOR TEACHING STUDY SKILLS 

v 



/ .2.36 
/ 2.80 
/ 2.81 
/ '2 .87 
*/ 2.67 
/ 2.47 

f 

/ 2.67 



5 MSEC/SEC 
STUDENT , 

P.'O'.S, 
ALL CAI 

FILES . . * : " 
P.Q.S, / 
ALL' CAI 
FILES 
P.Q.S, 
ALL CAI 
FILES: 
P.Q.S, 
ALL CAI 
FILES 
H,K,M»P, 
Q,S,X,ALL / 
CAI FILES f 
P,Q,S, 
ALL CAI 
FILES 

P.Q.S* 
ALL CAI 
FILES 



ERIC 



1 **2 

X ky 



v. 



FUNCTION v 



APPENDIX X (CONTINUED) 



S 



* * ' FILES 
/ VALUE NEEDED CPU TIME 



VB 



VB2 
VB3 
VB4 
' VB5 
VB6 
VB7 



VC 
V 



CAI CAPABILITIES 

» 

BASIC CAI CAPABILITIES 



BASIC CAI PLUS GRAPHIC CAPABILITIES 
BASIC CALPLUS CONSTRUCTED RESPONSE CAPABILITY 
BASIC CAI PLUS TOUCH PANEL OR- LIGHT PEN • 
BASIC CAI PLUS CONTROL OF AUDIO TAPE PLAYER 
'BASIC CAI PLUS CONTROL OF SLIDE OR FILM PROJECTOR 
BASIC CAI PLUS INTERFACE WIT# THE CMI DATA BASE 

FOR TRAINING ON LEARNING PROCESSES AND SPECIAL 
FOR STUDENT SKILL TRAINING 



/ 



/ 3.16 



CAI 
CAI 



<D V . VQ1 

(I)' VCZ CAI FOR INSTRUCTOR SKILL TRAINING 



/ 

' * / 
/ 

. / 
/ 

SKILLS / 
/ 
I 



2.74 
2.85 
2.92 
2.69 
2.74 
2.70 

2.62 
2.44 
2.49 



ALL CAI 
FILES 
CT 
CT 



5 MStC/SEC/ 
STUDENT "ON 
CAI r - 



x 



. 103 



1 



APPENDIX C (CONTINUED) 



FUNCTION 




/ VALUE 



FILES 
NEEDED 



CPU TIME 



(I) 
(I) 
(I) 
(I) 



#######' VI - C^II DATA BASE AND CAI MATERIALS PRODUCTION 
VIA' CMI CtiURSl DEFINITION. DATA BASE EDITIN&: 
.VIA! ON-LINE CMI DATA B*ASE EDITING * . 

VIA2 GRAPHICS FOR COURS£ STRUQTURE DISPLAY . * 
VIB AUTOMATIC VALIDATION OF COURSE 'DATA BASE 



'$) VIC , EDITORS FOR>PgOT)UCING ON-LINE TESTS 

VIC1 AUTOMATIC FORMAT FOR ^fjtRY %R;T£ST QUESTIONS 

VIC2 RANDOMIZED QUESTIONS AND* AL"$RNAT IVES « 

VIC3 CONTROL NUMBER OF ATTEMPTS J>ER QUESTIONS, 

VIC4 FEEDBACK FOR 1RIGHT/WRONG, IF DESIRED 

VID CAI. AUTHORING! EDITOR 

VIDl^PROVJDE AUTHORING EDITOR, NOT TEAM WRITING 
VI02 AUTHORING EDITOR FORMATS TEXT FRAMES AND QUESTIONS 
VID3] AUTHORING EDITOR STRUCTURES SEQUEllCE 'OF FRAMES , 
VID4UUTHOKING'£DIT0R COPIES FRAMES 'FROM OTHER LESSONS v 
VID5 AUTHORING EDITOR HAS USER "HELP" • INSTRUCTION? . 
VID6' AUTHORING EDITOR HAS "HELPS" FOR WRITING PRACTICES 



.VIE 
VIET 



VIE2 



USE AUTHORING E0JTOR FpR OFF-LINE MATERIALS 
USE AUTHORING ED t TOR Folt OFF-LINE TESTS 

USE AUTHORING EDITOR FpR OFF-LINE LESSONS 

. ( 



* 

\ 




'•er|c 
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NTENANCE 
/ 

/ 2.76 

/ 2.65 
/ 2.27 



/ 02 I,K 



/ 2.50 
/ 2.50 
/ 2.55 
/ 2.26* 



/ 

/ 2.50 
/ 2.48 
/ 2.49 
/ 2i53 
/ 2.38 
/ 2.47 

/ 2.50 
/ 2.58 
A 2.60 



####### * 

ALL FILES IN 
CMI DATA BASE. 
H » * 

C,F,H,K, 
L.M.P.R, 
X,W v 

r Vv 

; 15 MSEQ 
— AUTHOR i 

I,K 

UK 

AL1 CAI FILES 
ALL "CAI FILES 



;ec/ 



ALL CAI 
ALL CAI* FILES 
ALL CAI FILES 
ALL CAI XILES 
.ALL CAI FILES 
ALL CAI FILES 



20 MS/S/AUTHOR 
FILES 



CL.CO, 
CT 

P-,ce, 

CT 

CL.CO, 

CT * < 



20 MS/S/AUTHOR 



X 



Y%. APPENDIX C (CONTINUED) 



) 



FUNCTION 



FILES 

/ VALUE NEEDED CPU TIM£ 



VIE3 ' AUTHORING- EDITOR, JO REVIEW/REVISE OFF-LINE MATERIALS 
f Vl£4 PRINT MASTER COPY FOR OFF-LINE MATERIALS 
VIE5 PRINTS MULTIPLE COPIES FOR TRYOU1 
VIE6, -CONDUCT ON-LINE TRYOUTS 

.VIF USE EDITORS FOR COURSE DOCUMENTATION 



VI I* - INFQRMATION AND REPORTS ####### 
VIIA REPORTS FOR INSTRUCTORS' 
VIIA1 LEARNING CENTER ROSTERS 
VI IA2 BLOCK PROGRESS REPORTS 
VIIA2A LESSONS STUDENT HAS COMPLETED- 
VI I A2B ATTEMPTS AND SCORES ON LESSONS, 
VIIA2C OBJECTIVES FAILED IN BLOCK 
VIIA2D TIME ON LESSONS AND "ON BLOCK 
VIIA2E TIME AHEAD OR BEHIND SCHEDULE 
VIIA2F HOMEWORK ACCOMPLISHED 



/ 


2.51 


CL,CO, 








CT 




/ 


2.35 


CL,CO, 








CT . 




/ 


2.37 


CL,CO, 








CT 


* 


t 


2.28 


ALL CAI 








FILES 




/ 


2.18 


d,co, 


15 MSEC/SEG/. 






CT 


ON-LINE USER 



/ 2.32 


M,P,S 


f 2.41 


m,p;s 


/ 2.25 


M»S 


/ 2.37 - 


m,s; 


/ 2.37 


M,S 


7-2.16 


• M,S 


/ 2.31 


M,S 


/ 2.49 


M t S 


/*2.66. 


M,P»S 



: APPENDIX C (CONTINUED) 



(S). 
(S) 

(s) 

(S) 
(S) 
(S) 

(S) 



FUNCTION 



/ 



t 



VI I A3 STUDENT HISTOR/ REPORTS 
VIIA3A ATTITUDE AND INTEREST INFORMATION 
VIIA3B SCORES ON ENTITY SKILLS 
YIIA3C TIMES IN BLOCK REMEDIATION 
VIIA3D OBJECTIVES FM LED MORE THAN TWICE 
VIIA?E ABSENCE TIMES AND REASONS, BY BLOCKS 

VII A4A ADDITIONAL REPORT: ABSENCE REPORT 
VIIA4B ADDITIONAL REPORT: HOMEWORK SUMMARY 
.VIIA4C ADDITIONAL REPORT:. ON-LINE DISPLAY OF S 

vVlB -TRAINING MANAGEMENT REPORTS 

REPORT, STUDENTS AWAITING TRAINING 
REPORT, STUDENTS BY BLOCK, ROOM, OR BASE « 
REPORT, DISTRIBUTION OF BLOCK TIMES 
REPORT, DISTRIBUTION OF. COURSE TIMES 
REPORT, PERFORMANCE BY ROOMS OR BASES 
REPORT, RESOURCE UTILIZATION 
REPORT, INSTRUCTOR UTILIZATION/PERFORMANCE 
REPORT, STUDENT TIMES AND GRADES BY BLOCKS 



vriBi 

VIIB2 
•VIIB3 
VII-B4 
VIIB5 
VIIB6 
VOB7 
VIIB8 



/ VALUE 



FILES 
NEEDED 




/ 2.42 
/ 2.82_ 
t 2.47 
/ 2.59 
«/ 2.34 
f 7 2.62 

/ 2.58 
• / 2.79 • 
/;2.31 

7 

* /. 2.76 
/'2.63 
/ 2?. 70 
/ 2.71 
/ 2.95 
/ 2.64 
7 2.54 
/ 2.60 



M.P.S 
H.P.S 
M.P.S 
M,P,S 
M.P»S. 
M,P,S 

A,P,S 
M,P S S 
M,S 

E,S 
E.P.S 
RDF,V 
RpF,V 

E. P.S 

F, B 
RDF,V 



APPENDIX C (CONTINUED) 



/ 



FUNCTION 



• FILES - 
/ VALUE NEEDEfi CPU TIME 



S) 
S) 
(S) 
IS) 
(S) 

ts) 

sr 

(S) 
(S) 

;s) 

a- 

Si 

S 

;-s) 
;s) 

s ■ 

;s) 

V 

:s) 



VIIC 
VIIC1 
VIIC2 
VIIC3. 
VIIC4 
VIIC5 
VIIC6 
VIIC7 
VIIC8 
VIIC9 
•VIIC1Q 
VIIC11 
VIIC12 

VIID 

'viim 

VIID? 
VIID3 
VI 1 04' 
V1ID5 
VIID6 

VI 107: 

VI 1 08 
VIID9 

VI IE 
VIIE1 



COURSE EVALUATION SUMMARY 

FIRST ATTEMPT TIMES AND SCORES . 

^.lif firs? St'times and score; . 

MEAN, SO FINAL LESSON AND BLOCK SCORES^ 

MEAN SD FIRST ATTEMPT, LESSON AND BLOCK, TIMES • 

MEAN SD LESSON AND BLOCK COMPLETION TIMES 

SCORE AND TIME DATA BY LESSON ALTERNATES 

SCORE DATA BY OBJECTIVES IN LESSONS 
FIRST ATTEMPT LESSON FAIL RATES 

FAIL RATES ABOVE »X» PERCENT 

TEST ITEM EVALUATION REPORT 
TIMES EACH ALTERNATIVE IS SELECTED . 
TIH^-oSI QUESTION IS ANS UERED CORRECTLY 
FOR ALTERNATIVES, AVERAGE TEST SCORE 

ffifc Alternatives missed by u or more 

ITEM REMAINDER CORRELATIONS ' AKin • 
MEAN. AND SD OF SCORES BY OBJECTIVE AND* TEST 
DISTRIBUTIONS OF SCORES ON TESTS ^J , ' 

RELIABIUTrCOEFn 

3S QUESTION ALTERNATIVES 



VIIE2 CAI, PERCENT SELECTING BY ATTEMPT NUMBER 



/ 2.52 
/ 2.62 
/ 2.53 
, / 2.96 
/ 2.93 

V 2.94 
>/ 2.89 

7 2.88 
/ 2.68 
* / 2.70 
/ 2.67 
7 2 .80 
/ 2.72 

/ 2.23 
/ & 27 

V 2.20 
/ 2 .44 

^/ 2.07 
' / 2.50 
/, 2.57. 
/ 5.39 
/ 2.57 
7 2.40 

: / z& 

or* 

/ 2.70 



Z 
Z 

Z- 

z, 
z 

z * 

z 
z 

z. 

z 
z 
z 
z 

z 
z 
z 
z 
*z 
z 
z 
z 
z 

'Z 



CRP,CB, * 
XD,CL»C0 

crp;cb, 

CD.'CL.CQ 



ERIC 
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APPENDIX C'.(COWH'NUED) 



1 FUNCTION 



ts) 

(S) 

> 

(S) 



VIIE3. CAi ; TIMES TO READ AND "TO RESPOND 
VIIE4 sCAi; UST UNANTICIPATED RESPONSES'! 
HI |E5 *.CAI , ^PEN-ENDED' STUDENT COMMENTS' X 
#iI£6^%AI, RECORD STUDENT PATH THROUGH LESSON 




E7 CAI", STATISTICS ON MAJOR UNITS OF PROGRAM 



(S) VII F • COURSEWARE' DEVELOPMENT MANAGEMENT REPORT^ / 



8 



DROP [sj 
DROP (S) 
DROP (S) 

(Si) 

. DROP (sr 

DROP (S) 

DROP (S) 
DROP (S) 

(S) 



\ 



IIG ' PERFORMANCE FORECAST REPORTS TO FIELD * , 

n&l PRE-SERVtCE EDUCATION, VOCATIONAL TRAINING 

VIIG2 PRE-SERVICE QCpeATIONv^tflB EXPERIENCE „ . « : 

WG3 AF PERSONNEL RfetOROl AND DATA - 'ASVAff, \rtl 

.VIIG4 IN-SERVICE TRAINING PERFORMANCE DATA 

,VIIG5 SPECIALTY-RELATED DEFICIENCIES 

VIIG6 PREDICT PERFORMANCE. IN FIELD ASSIGNMENT 

VIIH SPECIAL RURPQSE REPORTS • : * 

VIIH1 SPECIAL REPORTS " BASIC DATA RETRIEVAL*, BACKGROUND 

VHH2 SPECIAL REPORTS - INTERACTIVE RETRIEVAL .• 

VIIH3 SPECIAL REPORTS - RETRIEVAL WITH STATISTICS 



,. . ^INTERACTIVE FROM DISC 
....BATCH ONLY ON-SHIFT 
....BATCH ONLY OFF-SHIFT * 



V 



' FILES 
7 VALUE NEEDED 



CPU TIME 



/ 2%88 .CRP.CB, • \ 

,CD»CL,£0 
7 2.8J * CRP,CB, ' v 

... CD>QL s CO V 
/ 2.82 CBiCCCD, 

CL.CO' - . 
/*2.70 CRP.CR, ' ' *, • . 

CD^CL.CO 
i 2.65. CRP,CB, 

^ CD,CL,CO - J 

/ 2.^9 ; " ... 1 

/ 2. 71 S '* • * . " 

/ 2.82 S 

/ 2.92 S. * 

/ 2.93 S* •* 

/ 2.87 S 

2.72 S V * 

/ 2.92 S 

/ 2.99 BvV.RDF 
/ 2.83 B,V,I?DF 
/ 2.74 B,V,RDF 20 HSEC/SEC/ 

ON-LINE USER 

,B,V,RDF 
B,V,RDF 
B,V,RDF • ' 



Abbreviations:- cv 

s 
b 

9 



APPENDIX C (CONTINUED^) 

* course version , LC * Vearning center 

= student \ * , * L - lesson ; . 

« Mock ' ' • M ■ module 
= group* (group or block) " " 



File 



CAI RECORDS 



Number of Records 



Max, Size 

(Words) Size (if variable length] 



CL CAI lesson recof.d * ' 
CO CAI objective record 
CB Branching record 



CT Text record, -type 1 



CRP Response point record 

CD Decision point record 

CC Comment record 

CSL Restart record - * 

CSR Restart record - 



1 /module with CAI 

1 /objective 

1 /objective that has , 
branching (some branch- 
ing instructions 
require 3 words) « J 

1/text frame 



type 2 1/text frame 



1 /student response 
1 /decision poijit passed 
1 /comment recorded 
1 /student taking CAP lesson 
1 /student in CAI review/ 
remediation • 



257 
623 
1012 



504 

506 

40 
35 
512 
64 

270. 



57 + (# of highest objective x 2\ 
23 + (# of frames x 61 ; 
12 + (# of branches)/ .* 



4 + text buffer, Average length 

25 wor$ls / 
6 + text buffer J average length . 

50 words 

16 + (# of unant|ci*pated responses) 
15 + (# of decisions/frame * 2) 
12 *+ buffer, average ■ 30 words 

1 * 

65 + {(# of objectives +^4)/5) 



11° . 



in 



9 

ERIC 



o 



1 if> 

ERIC 



File 



Number of Records 



APPENDIX C (CONCLUDED) 

Max. Size 
. , (Words ) 



Sflze (if variable length)' 



A 


Absence 


1/S/b if absent in block * 


511 


B 


Block 


1/s/b completed this week 


. r 


• C 


Course 


1/cv v 


606 


D 


Student Data Profile. * 


1/s usina file 




• 


Vist of lessons to do 








Course Enrollment 


1/S S s> 


i 


F 


Class t Facility) 


2/LC 


242 


H 


Hierarchy of Course 


i/g 


"284 . 


f, 


Item Record ^on- line 


♦ 






testing) 






K 


Test Key 


1/test" 


254 


L 


Lesson 


1/L 


14* 


M 


Module Completion 


1/s/started block 


508 




Record* 






N 


Module 


i/m • * 


222 


.P 


Learninq Center 


1/LC/shift/ 


155 


Q ^ 


Queue 


366 .( 


509 


R 


Resource Type DesclMp. 


1 * • ' 


801 


S 


Student Status 


1/s ; . , , 


452 


T« 


Testing Record 


1/s taking test 


260 


V 


Variables Definitions 


1 /variable 


13 


W 


Calendar 


T/cv 


288 


X 


Resources Cross-Ref- 


Vb 


156 




erence 






z 


,CMI 


1 /transaction '- 


29 


RDF 

<* 


Recent Data File 

*» 


400/b/course 


'688 




# m 

* * 


* » ' 

« * 





1 



p + {# of absences x_ 2) - 

♦ 

* * » 

I 2 + (4 x last type defined) 
16F + (3 x # of nodes) * - 



4 



7 + # of subscales + # of items' 
in each subscale 

w 

88 + (3 x #'of transactions) • 
206 + # off resources t 



+ # of res?ur< 



422 + # of resources 
11 + Xof itejns answered 



6100 + # of highest tesson taken 



. ' f APPENDIX D ' t ' • _m • 

ORDERED LISTING OE-\CAI/CMI FUNCTIONS, ACCORDING TO AVERAGE RATING FROM ATC RESIDENT TECHN^AL TRAINING 
PARTICIPANTS, WITH DISTRIBUTIONS' OF RATIIfi RESPONSES AND PROPORYlONS OF HIGH (1 AND 2) RATINGS 

Items are designated as (I), (S), or blank.* The (I) items appeared. only on the instructor/ 
1 survey form, the (S) items -only on the supervisor form, and the blank items appeared on 
both the instructor and supervisor foiwis.* 

- NOTE: 



ITEM 



These means were derived by averaging all of the individual responses, and are 
therefore somewhat different from the Appendix C means, which were derived by^ 
averaging the group averages. 7 > t . .* 

V FREQUENCIES, CATEGORIES 
r I . 1-5 AND X " 
• < - ' . ■ * - 1 1 3 '4,5 "1 MEAN 



PROPORTION 
*0F 1 & % 
RATINGS 



(S) 

(I) 

ft) 
(I) 
(I) 
'(S) 
(S) 

(I) 

(s) 

(I) 



mm FLAG ALTERNATIVES MISSED BY U OR MORE * / 43 

I A TEST SCORING AND FEEDBACK / 86 

IIF1 ASSIGN SUPPLEMENTARY TRNG BASED ON ENTRY SKILLS / 50 

VIIA2C OBJECTIVES FAILED IN BLOCK / 76 

•VIID2 TIMES EACH QUESTION IS ANSWERED CORRECTLY / 

I AT RIGHT-WRONG WITH PERCENT CORRECT - • / 

IIF2 ASSIGN SU?>PL. TRNG BASED ON LEARNING SKILLS / 47 

IB " ADD'L ON-LINE TEST CAPABILITIES / 

VIID TEST ITEM EVALUATION REPORT / 

* VIC4 FEEDBACK FOR RIGHT/ WRONG, AS SELECTED / 

VIF USE EDITORS FOR COURSE DOCUMENTATION 

VI 101 TIMES E'ACH ALTERNATIVE IS SELECTED 

VHA2 BLOCK PROGRESS REPORTS * 

I IF *" ASSIGN TO SUPPL. SKILL TRAINING ON ENTRY 

IA5 PASS SPECIFIC* OBJECTIVES 

VIB AUTOMATIC VALIDATION OF COURSE DATA BASE ' * 

VIE6 * CONDUCT ON-LINE TRYOUTS 

VIIA4C ADD'L REPORT: ON-LINE DISPLAY OF STUDENT DATA 

VIIA2D TIMES ON LESSONS AND BLOCKS . * / 68 



35 
46 



36 
24 
36 
I 77 
/ 33 
7 72 
/ 73 
/ 42 
/ 61 
63 
75 



/ 
/ 



43 28 10 1 5 2.06 
Bl &5 21 ' 7 17 2.13 
36" 25 7 10 9 2.15 
91 59 19 , 9 13 2.19 
45, 33 10 2 5 2,19 
30* Jl >2 5 13 2.19 
35 m 8 11 «11 2.21 

44 32 12 2 4 2.21 
35 33 5 1 32 2.22 
40 23 10 7 21 2.24 
66 48 19 16 *4> 2.25 
44 34 11 3 5 2.26 
70 78 17 10 20 2.28 

83 53 24 16 18 2.31 
34 27 12 10 12 2.31 

84 57 22 12 31 2.32 
80 61 22 11 30 2.32 
76 64 25 15 12 2.33 
78 73 28 8 12 2.33 



.-69 

.67 

.67 

.66 

.64 

.61 

.65 

.63 

.60 

•66 

.63 

.62 

.57 

.63 

.61 

.61 

.60 

.59 

*57 



APPENDIX D (CONTINUED) 



ITEM 



FREQUENCIES, CATEGORIES 

1 -.5 AN1TX ; 
"~T 2 3 4 § X 



PROPORTION 
' .O F 1 & 2 
MEAN RATINGS 



(I) 
(S) 
(S) 

(ir 



2 



I) 
I) 

s 

(I). 



(V) 



■81 



1) 
s) 



(i) 



VI I A REPORTS FOR INSTRUCTORS ' / 67 

IC3 LEARNING PROCESS SKILLS • / 37 

VIID9 RELIABILITY COEFFIC FOR OBJECTIVES AND TESTS / 27 

VIID7 DISTRIBUTIONS OF SCORES ON TESTS * / 26 

IIQ1 ASSIGN TO ALTERNATE LESSON REMEDIATION * 33 

VIIA3D OBJECTIVES FAILED MORE. THAN TWICE - /'70 

IV$ , IDENTIFY STUDENTS NEEDING SPECIAL ATTENTION / 64 

IVB IDENTIFY MARGINAL PERFORMANCE IN COURSE / 6V 

IVB3 . FLAGGED FOR ELIMINATION FR0M- TRAINING / 38 

IVB4 FLAGGED FOR SPECIAL REMEDIAL TRAINING / 33 

VIC • EDITORS .FOR PRODUCING ON-LINE TESTS • . / 22 

IC2 PRE-COURSE KNOWLEDGE TESTING > / 34 

VIIA2A LESSONS STUDENT HAS COMPLETED V > / 67 

VIE4 PRINT MASTER COPY FOR OFF-LINE MATERIALS / 60 

VIE5 PRINT MULTIPLE COPIES FOR TRYOUTS ' / 66 

VI I A2B ATTEMPTS AND SCORES ON LESSONS / 63 

IC4 COURSE-SPECIFIC ENTRY SKILLS * / 36 

IB1 ON-LIN#WITH RANDOM QUESTION ORDERS / 37 

IIIA SCHEDULE SltftNT ENTRIES TO COURSE / 6a 

VID5 CAI AUTHpR" MS "HELP" FRAMES WITH" EDITORS " / 44 

IB2 ON-LINE, RANDOM ORDERS FOR* ALTERNATIVES * / 36 

IID2 ASSIGN TO ALT. LESSON GROUPS REMEDIATION / 32 

VA CAI APPLICATIONS, TRAINING AND TESTING / 55 

VIIA1 LEARNING CENTER ROSTERS V ■ / 72 

IA4 PASS CERTAIN PERCENT OF OBJECTIVES / 36 

VIID3 FOR ALTERNATIVES,. AVERAGE TEST SCORES . / 26 

VIIA3 STUDENT HI ST,0RY REPORTS x / 53 

IIE2- TEST ASSIGN. EXCLUDES PREVIOUSLY TAKEN VERSIONS / 34 



70 , 82 18 11 19 .2.34 ' .55 

39 ' 31 10 10 JO 2.35 .60 

37 3fr 9 4 17 2.35 J7 
43 45 8 .3 5 2.35,, .55 
4T 32 6 11 14 2.36 .60 
82 58 27 16 14 2.36 . 60 
84.-56 34 11 19*2;37 .60 
91, 58 , 29 13 15*- 2.37' .60 

38 28 12 11 10 2.37 , .-60 
41 32 10 9 12 2.37 .59 
47 39 11 2 9 2.37 .57 

38 37 12 7 9 2.38 • .56 
77 §4 34 12 13 2.40 .57 

76 64 22 16' 29 2.40 .57 
66 65 26 . 15 29 2.40 .55 
85 60 33 14 12 3.41 ,58 
35 34 12 10 10 2.41 .56 
30 35 10 IT 14 2.41 " .54." 

77 50 22 25 25 2.42 .60 
85 54 23 12 49" 2.42 * *59 
30 36 11 10 14 2.42 .54 

39 29 10 12 15 2.43 .58 
73 65 ,25 13 36 2,43 .55 
60* 72 36 12 15 2.43 .52 
35 33 1,1 13 9 2.45 . .55 
38 40 17 3 6 2.46 .52 
70 76 27 12 29 2.47 .52 
37 22 12 15 17 2.48 ' .59 
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(I) 
(I) 

(s> 

(S) 
(S) 

-<s) 
(s) 

(I) 
(I) 



IVA2* 
VIIA3B 
IV A3 
VIIEl * 
IC1 
VIC2 

iiib 

VID2 

VC1 

VICT , 

VA5 * 

VID4 

VID3 

VID1 

VIIB7 

V1IA2E 

IC 

IID3 

IVA1 

VIID5 

VIID8 

VID6 

VIIB8 

VIIC 

V1ID6 

IIIB4 

IIB3 

VC2 

* 

J- 



PROFIT AND OEFIC. SCORES PRINTED OUT 
•SCORES Oft ENTRY SKILLS 

SCORES ON CRITICAL ENTRY VARIABLES PRINTED OUT 
' CAI, SELECTING EACH ALTERNATIVE 
RELEVANT AF PERSONNEL DATA 
RANDOMIZED QUESTIONS AND ALTERNATIVES ', 
GRADUATION FORECASTING , 
CAI AUTH. > FORMATS TEXT FRAMES ANB QUESTIONS . 
CAI: STUDENT SKILL TRAILING 
AUTOMATIC FORMAT FOR TESO QUESTION ENTRY 
CJU FOR BLOCK REMEDIATION . 
CAI AUTH. , COPIES FRAMES FROM OTHER LESSONS 
"H. , STRUCTURES SEQUENCE OF FRAMES , 
'HORING EDITOR, NOT TEAM WRITING 
'ORT, INSTRUCTOR UTILIZATION AND PERFORMANCE 
(HE AHEAD OR BEHIND SCHEDULE 
IEASSESSMENT DATA CAN ,BE USEI\ 
iSIGN TO BLOCK AND COURSE REMEDIATIONS 
LAGGED ON ENTRY VARIABLES 
ITEM REMAINDER CORRELATIONS ' 
ALPHA RELIABILITIES FOR OBJECTIVES AND TESTS 
CAI AUTH., "HELPS" FOR WRITINQ PRACTICES 
. REPORT, STUDENT TIMES AND GRADES BY BLOCKS 
COURSE EVACUATION SUMMARY 
MEAN AND SD OF SCORES BY OBJECTIVE AND TEST 
• DIRECT INTERFACE TO CBPO COMPUTER 
' ORDER BASED 0^1 RESOURCE AVAILABILITY 
. CAI: INSTRUCTOR SKILL TRAINING 

A~b1ank.in this column indicates that less than 
.rating, an. item assigned a .rating of 1 or 2. 



"FREQUENCIES, CATEGORIES 
V- 5 AND X 



T 



X 



X MEAN 



PROPORTION 
OF J & 2 
' -RATINGS* 



14 

8 
9 



/ 27 fl* 35 11 10 13 

/ 59 77 74 24, 21 12 

/ 32 t\ 37 , 14 9 

/ 20 42 44 f 1 5 

.. / 32 32 4"T*14 ;-9 

/ 23 42 27 14 ,' 9 22 

7 57* 71 54 28 22 35 

/ 41 7C 58. 28,, 12 54 

/ 24 38 32 ,15 ' 7 21 

/ 22 36 37 11 7 24 

/ 32 33 22 17 12 21 

/ 38 80 64 27 12 46 

/ 39 70 66 27 10 55 

/ 40 72 65 .23 14 53 

/ 24 40 36 16 T 7 

/ 58 72 74 32 18 13* 

/ 56 66 75 26 19 25 

/ 32 36 28 '8 18 15 

/ 28 39 28 18 .10 14 

/ 20 34 42 15 3 16 

/ 23 27 40 13 5 22. 

/ 44 75 50 33 16 49 

/ 22 40 33 ' 18 6 11 

/ 19 38 43 8 8 14 

/ 22 35 46 14 5 8 

/ 54 69- 43 40 19 42 

/ 28 38 30 19 1 0 12 

/ 26 36 29. 15 11 20 

ha'lf of the participants 



2.48 
2.49 
2.49 
2.50 

"2.50 
2.51 
2.51 
2.51 
2.51 
2.51 
2.52 
2.52 
2.5a 
2.53 
2.53 
2.53 

.2.53 
2.54 
2.54 
2.54 

•2.54 
2.55 
2.55 
2.55 
2.55 
2.56 
2.56 
2.56 



.55 

.53 

.51 

.51* 

.50 

.57- 

.55 

.54 

.53 

.51 

.56 

.53 

.51 

.52 

.52 

.51 

.50 

.56 

.54 



.55 

.52 



.55 
.53 
.53 
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IIIP3 
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APPENDIX D (CONTINUED) 



4 



FREQUENCIES 
1 - 5 



T 



> CATEGORIES 
AND X 



X "MEAN 



20 



CAI AUTM.,' USE TO REVIEW/REVISE OFF-LINE MAT 
MARGINAL TIMES AND SCORES PRINTED OUT 
CAI AUTH., USE_EQJM)FF-LINE MATERIALS 
DETERMINE 0Rt)ER OF ASSIGNMENTS . /* 
RESOURCE MANAGEMENT BY COMPUTER ' 
• CONTROL NO. OF ATTEMPTS PER QUESTION 
FINAL* TIMES AND SCORES 
REPORT, RESOURCE UTILIZATION 
ORDER BASED' ON LESSONS COMPLETED 
ASSIGN TO ALTERNATE REMEDIATIONS ' 1 

JMES^IN &LOCK REMEDIATION • . 

D'L REPORT: ABSENCE REPORT 
CAI, STAT. ON MAJOR UNITS OF AEROGRAM 
FLAGGED ON TIME AND SCQRE V/felABLES 

absence times and reasons ^ by blocks 
report, students by block, room,. or base 
cai auth., for off-line tests 
order based on earlier performance 
graduates: schedule out- processing' 
cai for teaching study skills 
cai auth., for off-line lessons*' - 
Weight questions differentially 
graduation reports to c6p0 
cai: learning process, special skills trng 
graphics for cojjjfce ' structure' display 
homework accomplished . 
attitudes and interests ■ . : 

* A blank in this column Indicates that less than half of 
the participants rating*-an item assigned a rating of for 2. 



»LS / 
/ 

J , 

/ 
I 

■{ 

I 
/ 
/ 
/ 
/ 
/ 
/ 
I 
I 
I 
I 
/• 
• / 
/ 
/ 

i 

7 
I 
I 



40 84 63 

31 31 35 

42 69 68 

42 7$ 80 
26 31 40 

23 35 31 
16 47 40 
25 „32 41 
30 30 41 
18 42 36 
53 64 82 
59 69 65 

21 27 56 

25 33 .42 
53 67 74 
20 43 36 
36 84 63' 

24 36 38 

53 58 55 

26 31 28 
36 82 60. 

22 41 34 
51 61 53. 

43 62 75 
45 S3 76' 

54 59 77 
47 58 79 



PROPORTION 
OF 1& 2 
.RATINGS* 
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.50 

.50 



.51 
.51 



.50 



.50 



.50 Ik-. 
.52 



..51 
.51 
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APPENDIX D (CONTINUED) 



ITEM 



FREQUENCIES, CATEGORIES 
1 - 5 AND X 



T 



PROPORTION 
OF 1 * 2 



2 3 4 5 X MEAN RATINGS 



VIIC1 FIRST ATTEMPT TIMES AND SCORES 

VIICIO FIRST ATTEMPT OBJECTIVE FAIL RATES ' 

VA4 CAI FOR BLOCK REVIEW 

VIIC8 SCORE DATA BY OBJECTIVES IN LESSONS 

VIIC9 FIRST ATTEMPT LESSON FAIL RATES 

IIC ASSIGN TO ALTERNATE, LESSONS' \ v <\ 

IIIB2 REPORT: GRADUATES IN NEXt N DAYS 

VUB3 REPORT: DISTRIBUTIONS OF BLOCK TIMES 

VIIH3 BASIC DAm RETRIEVAL PLUS STATISTICS 

VIIB4 RtPORT: DISTRIBUTIONS OF COURSE TIMES 

VHG5 SPECIACTY-RELATED DEFICIENCIES 

IVA4 PREDICTED COMPLETION TIMES. PROFIC/DEFIC 

VB7 CAI: INTERFACE WITH CMI DATABASE . 

VIIH2 BASIC DATA RETRIEVAL, INTERACTIVE 

IIG1 RESOURCE MGMT: ASSIGNMENT TO LEARNING CENTOS 
VI 1^12* SUMMARY, FAIL RATE ABOVE X PERCENT 

VIIE6 CAI, RECORD STUDENT PATH THROUGH LESSON 

IIC4 IN-COURSE PERFORMANCE DETERMINES ASSIGNMENT 

VIIE2 CAI, SELECTING BY ATTEMPT NUMBER i 

VB2 CAI, ADD GRAPHICS CAPABILITIES 

VB6 CAI , CONTROL OF SLITJE, FILM PROJECTOR 

VIIB1 REPORT, STUDENTS AWAITING TRAINING 

VUG PERFORMANCE FORECAST REPORTS TO FIELD 

IIIC4 MANAGEMENT TARGETS FROM WITHIN-COURSE DATA 

IIB4 . ORDER BASED ON TEAMING CONSIDERATIONS 

VA£ CAI TUTORIAL 

* A blank in this column~tndi£ates that less than half of the 
participants rating a& item* assigned a rating of 1 or 2. 
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47 


15 
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/ 23 
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34 


17 


12 


21. 


2.70 


/ 20 


28 


51 


17 
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2.70 


/ 20. 


27 


51 


17 
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2.70 v 


./ 38 


76 


67 


34 


24 


28 


2.71 


/ 25 


33 


26 


17 


16 


20 


2.71 


/ 18 


28 


52 


22 


'3 


7 


2.71 


/ 13 


27 


44 


13 


5 


28 


-2*. 71 


/ 17 


,35' 


41 


25 


5 


7 


2.72 


/ 20, 


30 


42 


19 


9 


10 


2.73 


/ 19 




41 19 


10 


16 


2.74 


/ 43 


-59 


65 


40 


23 


37 


•2.74 


/ 9 


30 


48 


13 
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26 


2.74 


/ 22 


27 


45 


16 


12 


15 


2.75 


/ 18. 


29 


52 


71 ; 


12 
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2.75 


/ 17 


29 


47 


21 
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10 


2.75 


/ 28 


28 


31 


17 


- 15 * 


14 


• f?K> 


V 15 


32 


47 


21 


6' 
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2.76 


/ 39 


71 


63 


37 


29 


28 


-2.77 


/ 45 


60 ' 


64 


45 


25 


28 


2.77f 


t 21 


.31' 


41 


20 


12 
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2.77 
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28 


43 


22 
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2.77 


/ 21 
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38 


18 


12 
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34 


24 


14 


19 
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2.79 
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'ITEM 



.FREQUENCIES, CATEGORIES 
* 1 - 5 AND X 



VB5 

VIA1. 

VIIA4B 

VIIF 

IIC1 

IIIB1 

IDT 

VIIA3A 

OA 

IIC5 

IB4 

IIC3 

VIIE4 

VIIE5 

VI 161 

ID2 

HE 

IIE1 

IIIC 

VI 164 

VA1 

VB3 

VIIC11 

VII63, 

VA3 

VIIE3 

-IIIC1 



CAI, CONTROL OF AUDIO TAPE PLACER 
ON-LINE CMI DATA BASE EDITING 
ADD'L REPORT: HOMEWORK SUMMARY , 
COURSEWARE DEVELOPMENT MANAGEMENT REPORTS 
INSTRUCTOR ASSIGNS ALTERNATES* 
PREDICTED' GRADUATION" DATES ON ROSTERS ' 
PRE 'AND POST- ATTITUDES 
ATTITUDE AND INTEREST INFORMATION ( 
ASSIGN TO ALTERNATE COURSE VERSIONS 
PRE-COURSE MEASURES DETERMINE ASSIGNMENTS; . 
ON-LINE, CONSTRUCTED RESPONSES 
RESOURCES DICTATE ALTERNATE ASSIGNMENTS 
CAI^ LIST UNANTICIPATED RESPONSES 
* CAI , OPEN-ENDED STUDENT COMMENTS 
PRE-SEfcVICE EDUCATION OR VOCATIONAL TRNG 
WITHIN-COURSE ATTITUDES AND INTERESTS 
TESTS SELECTED AND ASSIGNED 
TEST- ASS I GNMENT AT RANDOM * 
MANAGE STUDENT" TO TARGET 
IN-SERVICE TRAINING PERFORMANCE DATA 
CAI DRILL AND PRACTICE 
CAI , ADD CONSTRUCTED RESPONSES 
CORRELATIONS, ACTUAL WITH PREDICTED TIME, SC( 
AF PERSONNEL -DATA - ASVAB, ETC. 
CAI SIMULATION 

CAI, TIMES TO READ AND TO RESPOND 
MANAGEMENT TARGETS PRINTED FOR ENTIRE COURSE 

A blank in this column indicates that less than 
participants rating an item assigned a rating of 
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/ 41 


59 


66 


41 


26 


34 


2.79. 
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46 




39 


18 


62 


2.79 • 
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57 


87 


36 


28 


15 
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47 


18 
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2.79 
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39 
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12 


14 
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/.23 


31 
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14 


20 


19. 


2-81 


/ 2J2 


Oft 

28 


38 


24 


13 


12 


2.82 


/ 43 


59 


78 


45 


27 


15 


2.82 


/ 40 


^ o 
63* 


66 


33 


35 


30 


2.83 


1 23 


28 


37 


14 


20 


15 


2.84 




O 1 

21 


29 


16 


23 


19 


2*85 


/ 23 


25 


39 


19 


17 


14 


2.85 


/ TO 

/ 13 


27 


5> 


20 
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11 


2,85, 


1 1 o 

/ 13 


27 


55. 


17 


9, 
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2.85 


/ 14 


32 


45 


26 


8 


5 


2.85 ' ! ' 


/ 23 


26 


40 


19 


18 


11 


2.87 


/ 35 


55 


84 


39 


27 


27 


2.87 


/ 20 


36 


29 : 


18 


21 * 


13 


2.87 


/ 27 


66 


90 


32 


28 


24 


2 • o7 « 


/ 15 


26 


45 


26 


7 


11 


2.87 


. /* 23 


23 


26 4 


16 


20 


29 


2.88 


/ 33 


60 


70 


43 


27 


34 


2,88 


*E / 15 


23 


58 


15 


12 , 
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2*89 


■ / 17 


32 


39 


18 


17 
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2.89 


/ 23 


19 


25 


23 


16 


31 


2.91 


t 11 


27 


49 


26 


6 


U 


2.91 


/ 18 


27, 


37 


21 


16 


18 


2.92 
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FREQUENCIES, 
1 - 5 



CATEGORIES 
AND X 



ITEM 



: f) 

(s) 
(i) 

(S) 
(S) 



if! 
8 

(i) 
(s) 
(i) 
(i) 
i) 

(6) 
'(I) 
(I) 

(I 1 
(I) 

(I) 



o 
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(I) 
(I) 

\\\ 



V4IC6 MEAN, SD FOR LESSON & BLOCK COMPLET. TIMES , 

VII62 PRE-SERVICE OCCUPATION & JOB EXPERIENCE 

IB5 ON-LINE, ADAPTIVE WITHIN TEST 

VIIH1 BASIC DATA RETRIEVAL, BACKGROUND MODE 

IIG3 MANAGE RESOURCES ACROSS LEARNING CENTERS 

VIIC4 MEAN, SD FOR FINAL LESSON AND BLOCK SCORES 

VIIC5 MEAN, SD FOR 1ST ATTEMPT • LESSON ,* BLOCK SCORES 

VB4 CAI, ADD TOUCH PANEL OR LIGHT PEN . 

VIIC3 MEAN, SD; 1ST ATTEMPT TIMES AND.SCORES / 

IIC7 » PERFORMANCE PREDICTION DETERMINES ASSIGNMENTS 

IIIC2 MGMT. TARGETS PRINTED FOR INDIVIDUAL BLOCKS 

VIIC7 SCORE AND TIME DATA BY LESSON ALTERNATES 

XIB5 ORDER SELECTED BY INSTRUCTOR ■ 

VIIG6 PREDICT ADAPT, PERFORMANCE IN FIELD ASSIGNMENT 

IIG2 RESOURCES: ASSIGN TO OPTIMIZE ACROSS BLOCKS 

IIE3 ASSIGNMENT TO ON- OR OFF-LINE TESTS 

IIIC5 MGMT. TARGETS, RELATIVE TO AVERAGE CRSE. LENGTH 

VIIB5 REPORT PERFORMANCES BY ROOMS OR BY BASES" ' 

IB3 ON-LINE, RETEST OVER FAILED OBJECTIVES 

IIIC6 MGMT. TARGETS, RELATIVE TO POJ LENGTH 

IB6 ON-LINE, ADAPTIVE ON tOURSE PERFORMANCE 

IA7 ' SCORE PERFORMANCE CHECKLISTS % 

IIIC3 f MGMT. TARGETS BASED ON PRE- COURSE, DATA- % 

VB1 CAi; BASIC CAI CAPABILITIES 

IA2 CORRECT SCORE FOR* GUESSING 

IA8 INSTRUCTOR CAN .CERTIFY LESSON PASS , . 

IA6 PASS SPECIFIC QUESTIONS 

IIC6 HEURISTIC RULES FOR ASSIGNMENT SELECTION 

IIC2 STUDENTS SELECT -DESIRED ALTERNATIVES 

* A blank, in this column indicates that less than 

* participants rating an item assigned a rating of 
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APPENDIX E 
-SURVEY OF CAI/CMI FUNCTIONS— 



4i 



Listing of functions and mean ratings from Air Fore's training other than resident technical: Fie 
training detachments/on- the- job training (FTD/QJT); Undergraduate Navigator Training (UNT); Air 
University (AU); and HQ AT$ O0, P^lot Training (PT). 



Id 



Items are designated as (I), (S)j or blank. The (I) Items appeared only on the instructor survey 
form; the (5) items appeared only on the supervisor forms i and the blank i tems^ appeared on both in- 
structor and supervisor forms. . , 

* ♦ 

One supervisory , participant from /CTC HQ DO (Pilot Training) represented the yiews of thai organ- 
ization. The participants from Air University were all supervisory personnel. From these locations* 
there are no ratings for the items which were only on the instructor survey form. . > 



FUNCTION 



, / FTD/OOT 



WEAN RATINGS 
UNT PT Mi 



I - STUDENT TESTING AND EVALUATION ################################################################# 



IA TEST SCORING -AND FEEDBACK 

(I)'IA1 RIGHT-WRONG WITH PERCENT CORRECT 
(I) IA2 * CORRECT SC0R£ FOR GUESSING 

(I) IA3 WEIGHT QUESTIONS DIFFERENTIALLY 

(I.) IA4 PASS CERTAIN PERCENT OF OBJECTIVES 

(I) IAS PASS SPECIFIC OBJECTIVES 0 
(I) IA6 . PASS SPECIFIC 'QUESTIONS 

(I) IA7 SCORE PERFORMANCE CHECKLISTS * 

(I) IA8 INSTRUCTOR CAN CERTIFY LESSON PASS 

(S) IB ADDITIONAL ON-LINE TEST CAPABILITIES 

I) IB! ON-LINE, RANDOM QUESTION ORDERS 

IB2 ON-HNE, RANDOM ORDERS OF ALTERNATIVES 

IB3 ON-LINE, RETEST OVER FAILED OBJECTIVES n 

IB4 ON-LINE, CONSTRUCTED RESPONSES 

IBS ON-LINE, ADAPTIVE WITHIN TEST v 

IB6 ON-LINE, ADAPTIVE ON COURSE PERFORMANCE 



In 



2.08 


% 2.33* 


3.00 


2.42 


1.50 




2.17 


4.00 




2.33 


2.50 




2.08 


1 2.00 




2.00 


' 1.00 




3.00 


2.50 




2.83 


2.00 




2.58 


»3.50 




2.62 


2* 38* 


* 3.00 


2.17 


2.50 


If 


2.25 


2.50 


* 


2.58 


2.50 




, 2.83 


2.50 




2.90 


,2.00 




- 2.58 


2.50 





2.29 



2.89 



APPENDIX E (CONTINUED) 



FUNCTION 



IC P REASSESSMENT DATA CAN BE USED 

I) IC1 RELEVANT AF PERSONNEL DATA 

I) IC2 PRECOURSE KNOWLEDGE -TESTS 

il) IC3 LEARNING PROCESS SKILLS 

(I) IC4 COURSE-SPECIFIC ENTRY SKILLS 

ID ATTITUDES. AND INTERESTS 

(0 IDT PRE AND POST ATTITUDES 

(I) ID2 WITHIN COURSE ATTITUDES AND INTERESTS 

II - STUDENT ASSIGNMENT 

IIA ASSIGN TO 'ALTERNATE COURSE VERSIONS ' 

MB DETERMINE ORDER <3F ASSIGNMENTS 

(I) IIB1 ORDER BASED ON LESSONS COMPLETED 

(I) IIB2 ORDER BASED ON EARLIER PERFORMANCE 

(I) IIB3 ORDER BASED ON RESOURCE AVAILABILITY * * 

(I) IIB4 ORDER BASED ON TEAM CONSIDERATIONS 

(I) IIB5 ORDER SELECTED BY INSTRUCTOR 

IIC ASSIGN TO ALTERNATE LESSON TREATMENTS 

ill IIC1 INSTRUCTOR ASSIGNS ALTERNATES 

(I) IIC2 STUDENT SELECTS DESIRED ALTERNATES 

(I) IIC3 RESOURCES DICTATE ALTERNATE ASSIGNMENT 

(I) IIC4 IN-COURSE PERFORM. DETERMINES ASSIGNMENT 

(I) IIC5 PRE-COURSE MEASURES DETERMINE ASSIGNMENT- 

(I) IIC6 HEURISTIC RULES FOR ASSIGNMENT SELECTION 

I) IIC7 PERFORMANCE PREDICTION DETERMINES ASSIGNMENT 

S) IID«v ASSIGN TO ALTERNATE REMEDIATIONS 

(I) I ID! ASSMSN TO ALTERNATE LESSON REMEDIATION - 

(I) IID2 ASSIGN TO ALTERNATE LESSON GRPS REMEDIATION 

(I) IID3 ASSIGN TO BLOCK OR COURSE REMEDIATIONS 

HE TESTS SELECTED AND, ASSIGNED 

(I) IIE1 TEST ASSIGNMENT AT RANDOM 

(I) IIE2, TEST ASSIGN, |XCLUDES PREVIOUSLY TAKEN TESTS 



i 



MEAN RATJNGS 
.FTD/OJT ONJ__ r _PT_ AU 

i , 

3.35 3.17 4.00 

2.75 3.00 

2.58 ,2.50 te 

2.50 ,3.00 

2.42 2.00 

3.05 3,42 

2.58 2.50 

2.67 2.00 



0 QQ 


O CO 


3.00 


3.39 


0 Q7 
C.Jf . 


9 'AO 


0.00 


3.50 


o ci 
c.oo 


c.UU 






2.67 


. 2.50 






2.50 • 


1.50 






2.95 


Lop 






2.83 


1.50 






3.00 


1.92 . 


2.00 


3169 


3.20 


2.00 






3.35 


3.00 


1 


# 


2.95 


3.00 






2.85 


1.50 






2.65 • 


. 3.00 • 






2.60 


4.00 






.2.75 


,1.50 






•2.85 


2.50 


3.00 


3.29 


3.08 


4.00 






2-67 


3.50 






2.92 , 


3.50 






2.68 


3.00 


2.00 


3.89 


3*33 • 


4.00 






3f. 1 7 - 


2.50 







2.59 
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appendix e Continued) 



- ; MEAN RATINGS . ' 
FIJNCTigN__ / fW0JT - _^UNT _PT _AU 



(I) JIE3 

• HF 

(I) IIFJ 

(I) IIF2 

I) IIG 

I) IIG1 

I) IIG2 

!l) IIG3 



ASSIGNMENT TO ON- OR OFF-LINE TESTS ' / 3.20 3.50 

ASSIGN. TO SUPPLEMENTARY SKILL TRAINING ON ENTRY / 2.76 2 f 67 

ASSIGN. SUPPLEMENTARY TRAINING BASED ON ENTRY SKILLS / 2.25 1.50 

ASSIGN SUPPLEMENTARY TRAINING ON LEARNING SKILLS / 2.80 2.00 

RESOURCE MANAGEMENT BY COMPUTER / 2.35 2.00 

RESOURCES: ASSIGN TO LEARNING CENTERS / . 3.20 2.00 

RESOURCES: ASSIGN TO OPTIMIZE ACROSS BLOCKS / 3.45 2.00 

RESOURCES: MANAGE OUTSIDE LEARNING CENTER . / 3.30 l.*50 



4.00 3.39 



III - STUDENT PROGRESS MANAGEMENT #######4#4##M^ 

II IA SCHEDULE STUDENT ENTRIES TO COURSE 

II I& GRADUATION FORECASTING , 
(I) IIIB1 PREDICTED GRADUATION DATES ON ROSTER 
(I) IIIB2 GRADUATES IN NEXT N DAYS, LISTED ON REPORT 

IIIB3 GRADUATION REPORTS FOR CBPO 

IIIB4 DIRECT INTERFACE TO PERSONNEL SYSTEM 

1IIB5 GRADUATES: * SCHEDULE OUTtPROCESSING 

1 1 it MANAGE STUDENT TO TARGET 
(I) IIIC1 MANAGEMENT TARGETS PRINTED FOR WHOLE COURSE 
;i) IIIC2 MANAGEMENT TARGETS .PRINTED FQR INDIVIDUAL BLOCKS 
I) IIIC3 MANAGEMENT TARGETS BASED ON PRE-COURSE DATA 
;i) IIIC4 MANAGEMENT TARGETS, FROM WITHIN-COURSE DATA 
(I) IIIC5 MANAGEMENT TARGETS RELATIVE TO AVERAGE COURSE LENGTH 
(I) IIIC6 MANAGEMENT TARGETS, RELATIVE TO POI LENGTHS 



IVA 
(I) IVAT 
(I) IVA2 
(I) IVA3 
(I) IVA4 



IDENTIFY STUDENTS NEEDING SPECIAL ATTENTION 
FLAGGED ON ENTRY VARIABLES" . 
PROFICIENCY ANQ DEFICIENCY SCORES PRINTEp 
SCORES ON CRITICAL ENTRY VARIABLES PRINTED % 
PREDICTED COMPLETION IMES, PROFICIENT-DEFICIENT 



/ 


3.12 


2.00 




4.33 


/ 


3.05 


3.17 


5.00 


4.*33 


/ 


3. OS 


1.50 






/ 


2.67 


1.50 






/.- 


2.77 


3.*44 


♦5.00 


4,'n 




2.69 


* .3.33 


5.00 


3.88 


/ " 


3.15 


2.63 




4.22 


/ 


3.22 


3.33 


5.00 


3.59 


/ . 


2.67 


2.00 






/ 


3.00 


2.00 






/ - 


-3.00 


4.00 






/ 


3.03 


2.50 






/ 


3.00 


2.00 






/ 

*####■ 


3.00 


2.50 




mm 


/ 


mmm 

3.03 


2.38 


>####### 


• 2.79 


/ 


2.92 


4.00 






/ 


2.67 


4.00 






/ 


2.58 - 


3.00 






/ 


3.00 


4.00 







JL 3 3 
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ERIC* 



pi 
ill 
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(I) 



APPENDIX E (CONTINUED) 



FUNCTION, . • ' t 



MEAN RATINGS 



AU 



( 

IVB/ IDENTIFY- MARGINAL PERFORMANCE IN COURSE 
IVB1 . FLAGGED ON TIME AND SCORE VARIABLES 
IVB2 MARGINAL TIMES AND SCORES PRINTED OUT 
IVB3 FLAGGED FOR ELIMINATION FROM TRAINING 
IVB4 FLAGGED FOR SPECIAL REMEDIAL TRAINING 



/ 

7 
7 



2.91 
2.67 
2.75 
2.83 
2.67 



2.25 
1.50 
2.00 
2.00 
3.00 



3.00 3,09 



V - COMPUTER-AIDED INSTRUCTION (CAI) ##################/#f #############################*############## 



VA CAI APPLICATIONS, TRAINING AND TESTING . 

VA1 CAI DRILL AND PRACTICE ■ 

VA2 ^CAI TUTORIAL 

Vfc3 CAI SIMULATION (LIMITED" CAPABILITY) 

VA4 CAI FOR BLOCK REVIEW 

VA5 CAF F0R BLOCK REMEDIATION 

VA6 CAI SDR TEACHING STUDY 'SKILLS - 

(VB .CAI CAPABILITIES) 

VB1 CAI: BASIC CAI CAPABILITIES 

VB2 BASIC CAI PLUS GRAPHIC CAPABILITIES. * 

VB3 BASIC C*AI PLUS CONSTRUCTED RESPONSE CAPABILITIES 

VB4 BASIC CAI PLUS TdUCH PANEL OR'ilGHT PEN 

VB5 BASIC CAI PLUS CONTROL OF AUDIO TAPE PLAYER 

VB6 /.BASIC Plus control OF SLIDE, film PROJECTOR . 

VB7 BASIC PLUS INTERFACE WITH THE CMI DATA BASE 

VC CAI FOR TRAINING" ON LRNG PROCESSES, SPEC SKILLS 

VC1 CAI FOR STUDENT SKILL TRAINING . 

VC2 CAI FOR INSTRUCTOR SKILL TRAINING 



/ 
/ 
/ 
/ 
/ 
/ 
/ 



2.87 
3.08 
2.83 
2,92 
2.75 
2.92 
3.42 



2.42 
4.00 

1.09 

1.50 
UOO 
2.00 
2.50 



2.00 3.00 



/ 
/ 
/ 
/ 
/ 
/ 

, / 
/ 

' 7 
- 7 



3.12 
3.12 
2.85 
3,17 
2.82 
2.83 
2.93 
3.05 
2.58 
1.75 



3.58 
2.89 
2.67, 
2.44 
3.08 
2.33 
2.25 
2; 50 
2,50 
3.50 



5.00 


3.11 * 


4.00 


3.44 


3.00 


3*66 . 


3.00 


-3.88 


3.00 


3.44 • 


2,00 


3.55 


2.00 


3.77 


3.00 

* » 


3.00 



VI - CMI -DATA BASE AND CAI MATERIALS PRODUCTION AND MAINTENANCE ##################*#########»########## 



(VIA CMI COURSE DEFINITION DATA BASE EDITINO) 

VIA1 ON-LINE CMI DATA BASE EDITING 

VIA2 GRAPHICS FOR COURSE " STRUCTURE DISPLAY 



/ 
/ 



3.05 
2.73 



2.33 
2.22 



1.00 
2.00 



3.19 
3.22 
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APPENDIX E (CONTINUED) 





, yiB AUTOMATIC VALIDAJION OF COURSE DATA BASE * 

(S> VIC EDITORS FOR PRODUCING ON-LINE TESTS 

[li no AUTOMATIC FORMAT FOR ENTRY OF TEST QUESTIONS . . 

I) VIC2 RANDOMIZED QUESTIONS AND ALTERNATIVES 

I) VIC3 CONTROii NUMBER OF ATTEMPTS PER QUESTION 

I) VIC4 FEEDBACK FOR RIGHT, HR0NG,,IF DESIRED 

/ (VID CAI AUTHORING EDITOR) ...... 

VID1 PROVIDE AUTHORING EDITOR, NOT TEAM WRITING 

. VID2 AUTH. EDITOR FORMATS TEXT FRAMES, QUESTIONS 

VID3 AUTH. EDITOR STRUCTURES SEQUENCE OF FRAMES 

VID4 % AUTH. EDITOR COPIES FRAMES FROM OTHER LESION 

VID5„ AUTH. EDITOR HAS USER "HELP" INSTRUCTIONS > 

•VI 06 AUTH. ED. HAS "HELPS" EOR WRITING PRACTICES 

' VIE USE AUm. EDITOR FOR OFF-LINE MATERIALS 

VIE! USE AUTif. EDITOR FOR OFF-LINE TESTS 

VVIE2 .USE AUJH. EDITOR FOR OFF-LINE LESSONS 

VIE3 AUJH. EDITOR TO RE VI EH, REVISE OFF-LINE MATERIALS* 

VIE4 PRINT MASTER COPY FOR OFF-LINE MATERIALS 

VIES PRINT MULTIPLE COPIES FOR TRYOUTS 

VIE6 CONDUCT ON-LINE TRYOUTS 

VIF USE EDITORS FOR COUBSE DOCUMENTATION,* 



VII - 



INFORMATION RETRIEVAL AND REPORTS ###################### 



VI IA REPORTS . FOR. INSTRUCTORS 
VIIA1 LEARNING. CENTER ROSTERS 
VIIA2 * BLOCK PROGRESS REPORTS ' 
VIIA2A LESSONS STUDENT HAS COMPLETED 
VI IA2B ATTEMPTS AWfi" SCORES ON LESSONS 
VIIA2C OBJECTIVES FAILED IN BLOCK 
VIIA20 TIME ON LESSONS AND ON BLOCK 
VIIA2E TIME AH^AD OR BEHIND SCHEDULE 
VIIA2F HOMEWORK ACCOMPLISHED 



✓ 

MEAN RATINGS 
• / FtD/OOT ' UNT PT * AU 



/ 


2*25 


2; 17 


• 2.00 


2.69 


< / 


3.02 


2.188 


3.00 


•2.89- 


• /" 


2.45 


2.00 






/ 


2; 35 


2.50 






/ 


2.50 


2.50 








2.25 


2.00 

• 






• • f * * 

/ 


2.66 


. 2.55 ' 


2.00 


2.44 


/ . 


2.72 


2.67 


2.00 


2.66 


/ - 


2.67 


2.78 


2.00' " 


2.55 


. / * 


'2.74 


2.25 


2.00 


2.77 


/ 


2.62 


2.17 


.2.00 


2.33 


/ 


2.72 


2.17 


*2.00 


2.77 


/. 


5.71 ' 


3.42 


4.00 


2.62 


/ 


2.88 


: 3.67 


4.00 


2.66 


/ 


3.02* 


3.25 


4.00 v 


2.88 


/ 


2.84 


3.17 


4.00 


2.33 


/ 


2.74 


-fc. 92 


4.00 


1.77 


• / 


2.80 


6.00 


4.00 


3.11 


/ 


2.86 


3.67. 


4.00 


2.55. 


/ 


2.64. 


3.25* 


' 4.00 


2.44 



/ 


.2.70 


2.50 


3.00 


2.72 


/ 


3.11 


3.00 


4.00 


3.27 


/ V 




2.08 


3.00* 


3.59 


/ * 


3.19 


2.33 


3.00 


3.44 


/ 


3.02 


* 2.25 


3.00 


3»66 


/ - 


2.92 


2.08 * 


3.00 


2.77 


/ 


3.10. 


2.83 


4..00 


3.77 


/ 


3:10 


" 2.33 


3.00 


4.00 


/. 


3.22 


2.13 




3.66 



. • . '. ' * • . ' * 

\ ? - MEAN RATINGS- 
FUNCTION - v , / ^ FTD/OJT _UNT PT AU _ 

VIIA3 STUDENT HISTORY REPORTS ' * / 2.90 3.33 4.00 3.50 

VIIA3A ATTITUDE AND INTEREST INFORMATION ^ */ 3.05 4.13 — 2.63 

VI IA3B ! SCORES ON ENTRY SKltLS 7 2.81 - 3.33 4.00 3.29 

VIIA3t TIMES >IN BLOCK REMEDIATION , , / 3.14 * ' 3.Q0* 3.00 4.33 

VIIA3D OBJECTIVES FAILED MORE THAN .TWICE . A 2*66 2.42* 3.0O 2.88 

•VUA3E ABSENCE TIMES AND REASONS, BY BLOCKS * / 3.17 3.00 — 4.11 

VI IA4A ADDITIONAL REPORT: ABSENCE REPORT V 3.24 3.58 4.00 4.39 

VI IA4B ADDITIONAL REPORT: HOMEWORK SUMMARY • . / 3.21 -3.75 4.00 3.69 

VI IA4C ADDITIONAL REPORT: ON-LINE DISPLAY, STUDENT DATA ' / 3.02 2,50 3.00 2.90 
(V.IIB* TRAINING MANAGEMENT REPORTS) 



(S) VIIB1 REPORT, STUDENT AWAITING TRAINING 
(S) VIIB2 REPORT STUDENTS BY BLOCK, ROOM, OR BASE 
M VIIB3 REPORT, DISTRIBUTION OF 1 BLOCK TIMES 
M VIIB4 REPORT, DISTRIBUTION OF COURSE TIMES • 
S) VIIB5 REPORT, PERFORMANCES BY ROOMS OR* BASES 
S) VIIB6 REPORT, RESOURCE UTILIZATION 
S) VIIB7 REPORT, INSTRUCTOR UTR I ZAT ION-PERFORMANCE 
S) VIIB8 REPORT, STUDENT TIMES, GRADES BY BLO.CKS 
S) VlIC COURSE EVALUATION SUMMARY 
S) VIIC1 FIRST ATTEm TIMES^ SCORES, 
S)l VIIC2 FINAL TIMES, SCORES' 
S) VIIC3 MEAN, SD, 1ST ATTEMPT TIMES AND SCORES 
[S) VIIC4 MEAN, SD, FINAL LESSON, BLQCK SCORES 
;S) V1I^5 i MEAN, SD, 1ST ATTEMPT LESSON, BLOCK T$M£S 
S) VIIC6 MEAN, SD, LESSON r BLOCK COMPLETION TIMES 
S) VIIC7 SCORE, TIME DATA BY 'LESSON ALTERNATES* 
S) VI ICS S60RE DATA BY OBJECTIVES IN LESSONS 
S) VIIC9 FIRST ATTEMPT LESSON FAIL RATES 
(S) VIICTO . FIRST ATTEMPT OBJECTIVE FAIL RATE8 * » 
VIIC11 CORRELATIONS, ACTUAL/PREDICTED TIME, .SCORE 
VIIC12 SUMMARY, FAIL RATE ABOVE "X" PERCENT 



(S) 



# * * * 

/ 


2.71 


* • % * *> * * 

3.75 


4.00 


* 4*29 




2.78 


3 * 25 


3.00 


4.11 


/ 
/ 


3.17 


2.88 


3.00 




3.11 


2.75 




4| 7^ 


/ 


3.00 • 


3.13 


3.t)0 


3**9 
"3.59 


/ 


2.94 


2.88 


3.00 


/ 


2.70, 


2 #88 


3.00 


3.50 




2.79 1 


3*00 


3.0a 


2,79 




3.02 


'2.75 


3.00 


v 3.44 


J* 


3.20 


* 2.88 


3.00 


3*88 


/ 


3.16 


2.63 


3.00 


2.90 


7 


3,35 


* 2.75 


3.00 


3.69 


/ 


3* 35 


2.75, 


3.00 


3.50 


/ 


3 ♦ 38 


2.8% 


3.00 


3.88- 


7 


3.29 


2.8a*: 

2.88* 


3.00 


3.77 


/ 


3.26 


3.00 


4.22 


/ 


3.08 


2.63 ' 


3.00 


* 3.19 


/ ' 


3.?Q 


'2i88 


3.00 


;4M 


7 - 


3.15 


,2.75 


34)0 


•3*77 


/ 


3.36 


2.75 


3.00 


% 4.33* 


/ ' 


'3.*13> 


2.75 


3.00 


3.25 



*" ER1C • / 3^ 



FUNCTION 



/ FTO/OJT UNT PT , AU. 



S) 

s) 

s 
s 
s 

s! 

» 



s) 

s 

;s. 
s 
s 
s 

is 

s 
s 
s 



(S) 



TEST ITEM EVALUATION REPORT 
VIIOl TIMES EACH ALTERNATIVE IS SELECTED 
VIID2 TIMES EACH QUESTION IS ANSWERED CORRECTLY 
VIJD3 FOR ALTERNATIVES, AVERAGE TEST SCORE 
YIID4 FLAG ALTERNATIVES MISSED BY X% OR MORE 
VI IDS ITEM REMAINDER CORRELATIONS 
VIID6 MEAN, SD, SCORES BY OBJECTIVE AND TEST 
VIID7 DISTRIBUTIONS OF SCORES- ON TESTS 
VI IDS ALPHA RELIABILITIES FOR OBJECTIVES, TESTS 
VI ID9 RELIABILITY COEFFICIENT FOR OBJECTIVES, TESTS 
( VI IE CAI EVALUATION DATA) « «,«*«9««»»**»**»***«4 
VIIET CAI, % SELECTING QUESTION ALTERNATIVES 
VIIE2 CAI, % SELECTING BY ATTEMPT NUMBER 
VIIE3 CAf, TIMES TOH*EAD AND TO RESPOND 
VIIE4 CAI, LIST UNdfclCIPATED RESPONSES 
VIIE5 CAI, OPEN-ENDE* STUDENT COMMENTS ' 
Vt»IE6 CAI * RECORD STUDENT PATH THROUGH LESSON 
VIIE7 CAI, STATISTICS ON MAJOR UNITS OF PROGRAM • 
VI !F COURSEWARE DEVELOPMENT MANAGEMENT REPORTS 
VUG PERFORMANCE FORECAST REPORTS TO FIEfcD *■ 
^IIGUPRE-SERVICE EDUCATION, VOCATIONAL TRAINING 
VlfGr PRE-SERVICE OCCUPATION, JOB EXPERIENCE- 
V Ft G3 AF PERSONNEL RECORDS DATA - ASVAB, ETC. 
VIIG4 IN-SERVICE TRAINING PERFORMANCE DATA , 
VIIG5 SPECIALTY-RELATED DEFICIENCIES 

vna predict Performance in .field assignment * 

(VIIH SPECIAL PURPOSE REPORTS) i.. 

VIIHV SPECIAL REPORTS- BASIC DATA RETRIEVAL, BACKGROUND 
VIIH2 SPECIAL REPORTS- INTERACT! VE RETRIEVAL 
VI IH3 SPECIAL , REPORTS-RETRIEVAL , WITH STATISTICS 



# • • * * 



9 07 


a nn 




% 3.08 


3.00 


3.00 


2.99 


3.13 


3.30. 


3.49 


3.00 


3.00 


2.94 


2.50' 


3.00 


3.21 


2. -88 


3.00 


3.27 


2.88 


3.00 


3.05 


2.75 


3.00 


3.04 


3.00 


; -- 


3.23 


2.75 


— . 


3.20 


3.00 


3. 00 


3.28 


3.00 


. 3.00 


3.29 


3.13 


3.00 


, 3.29 


3.00 


3.00 


3.12 


2 • 88 


3.dO 


3.13 


• 2.88 


3.00 


3.35 


,2.88 


J.00 


3.31' 


3.^25 


3J0O 


3.15 


. 2. SO 




3.20 


2.50 




3.38 


4.00 




3*36 , 


3.75 




3.11 


3.25" 




2.90 


3.13 


3.00 


3.53 


2.75 

* 


- 3.00 


3.21 


2.25' 


2.00 


3.07 * 


1.63 


"1.00 


3.47 


2.25 


2 t 00 



2.4- 

2.1 

2.2 

2.1 

2.0C 

3*12 

2.09 

2.63 

2.66 

2.55 

.3.00 
3.44 
3.66 
3%* 3^ 
3.tH 
3.71 
3 * 3« 
3.1 
4.1 
3.7 
4.0 

4.2 
4,3 
4.1 

2.7 
2.7 



> APPENDIX F . ' , 

An X Indicates that a program expressed a need for a particular function Catoaor* l tun****™ a _ 

2ddTtl^i b ? * lr c0S V ystem ' Actions requ?re Sit *5°2tf ftoSSTbT^- • 

^ e !? the low - cost Wtem, and Category 3 functions are those which would reaul re con 
v siderab,le modification or addition to the low-cost system. • require con- 

i^S^ - r -~ 
SUMMARY: ^ tte jgjb 1 ties mentioned below, are already included in ^specification for the 
iow-cos^ CAI/CMI system. Item, 14 below. would require minor software modification to 
accommodate coding to STS, SkiflUevel, and CDC volume. Item 15 requires^ ^ nter?Sce between 
ft£ L£Z ?*™^J>™*™™ a " d loc *t1ons, and should be Invented Site fnteT 
face between the low-cost system and other AF computer based systems il being defined 

7~ — £— :r 19 -:- r — —*~ as ^ . LJSSLJtS&JSSJg&Xt /category 

1. On. lino foct i'4-om «. j .j-j ., < ---- , ....• M ,... a . w ( > .-a.*. 



1. 

2. 
3. 

4. 

5. 

6. 

7. 

8. 

9-. 

10. 

IK 

12.\ 
13. 



On-line test item development and modification 
Computerized storage of comprehensive test item banks . 
Association of .individual identification with training 
profile and next test . • , . ' ,my ., 

Verification of student eligibility for test ' 
Computer generation of next required test (includlnq 
retakes) * 3 

Test security *' ? * • • 

Remote, on-line testing capability * 
. Adaptive, branch testing" capability 
Rapid student access to required test * 
** P H ^ edDack °f test results to student and supervisors 
.Rapid distribution af 'test results and training 
comp/letions « ^ * • .* ; .* 

Continuous y>s% 1 tem anatysi s aVid feedback 
Ma intenjtffc" /{•storage)* of test data 
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APPENDIX F (CONTINUED) ' ' - * 

FUNCTION t . / S ECI FTD OOT UNT UPT /CATEGORY 

14, Computerized coding of test items to STS, Skill Level, . 

AFSC and CDC volume x 2 

15. Computerized elimination of duplicative, generic test „ _ 
Items between AFSCs X 3 

II - STUDENT ASSIGNMENT f \ . * J , , + M 

SUMMARY: Items 3 and 4 below would require minor modification to the low-cost system, and would entail 
much careful work by training personnel to provide the necessary tdefi nations and class- 
ifications so that these capabilities would be based on accurate and complete information; 



X X X X X 1 
X • X 1 



1. .Assignment of required training event, module, or volume 

2. Assignment of best available remedial material/ resources 

3. - 'Identify available/applicable study material in response - 

td student request X % X X X ? 

4. Preclude duplicative, redundant training .XX - 2 

I Il"-"sTUDENT PROGRESS MANAGEMENT ' • • . 

SUMMARY: Items 1 through 7 are included in the low- cost system. Items 8 through 13 involve interface 
with other programs or computer based systems, and should be planned as- part of that mter- 
k ' face. • 

1. - Associate individual identification with twining % • 

profiles % ^ ^ ] 

2. ' Provide training profile security " X X X X X 1 

3. Provide Individual procedural instructions t X X 1 

4. Establish individual progre§s goals/suspenses X X 1 

5. Notify of inadequate progress and/or missed suspenses .X X X v X 1 

6. Identify remedial needs on basis of test/qualification/ 
certification results 

7. On-line student sign-in/sign-out (includes flying, 

simulators, trainers) 

8. Continual assessment of individual training/ education 

requirements, based on: 
Civilian education and/or occupation(s) 
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APPENDIX F (CONTINUED) 



FUNCTION 



7 ECI FTD OJT UNT UPT /CATEGORY 



In-service training and/or education 
AFSC, Skill level, and STS 
Qualification and certification requirements 
Verification of Individual eligibility for self- 
requested training 1 
Verification of individual eligibility for 
computer-assigned training - 
Establish earliest and latest dates for retake 
of tests. 

Es tabl ish- and noti fy of qua! 1 f i ca ti on/ . 
certification suspenses » 
13. Prepare local training schedules (interface with 
host base maintenance schedule, provide weekly 
academic, simulator and flying schedules which can 
be adjusted daily for weather or maintenance aborts 
and test or flight check failures 



9. 
10. 

H. 
12. 
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l: On-line preparation of counseling reports 



/ 



/ 
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3 
3 
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IV - SUPPORT *F0R GUIDANCE AND COUNSELING 

SUMMARY: Minor modification needed to software to provide Item l^below. 



V - COMPUTER AIDED INSTRUCTION * . 

SUMMARY: Most of . the required CAI capabilities are Included in the low-cost system. Item 3 would 
require additional hardware (touch panels or light pens) and some software additions. 
Movement would require major hardware and software additions—and might best be provided 
by video tape or disk equipment treated as media devices, rather than as on-line CAI. 



1. • Interactive testing (excluding hands-on performance 

check items) 

2. " Interactive review and remediation 
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APPENDIX F (CONTINUED) \\ 
FUNCTION / ECI FTD OJT UNT ; UP"% /CATEGORY 




3. .Interactive (on-line) delivery of program (course) 

material - Programs include Type T (Contract Train- . « 
ing), Type 2 (Special Resident Training), Type 3 
(Regular Resident Training), Type 4 (Field Train- 
ing, FTD), Type 5 (Other Agency Training), Professional 
Military Education, On-The-Oob Training (OOT), 
Certification Training, Qualification Training, Re- 
qualification Training, and Ancillary Training. Should 
have touch panel or light pen, Graphics, Movement, ♦ 
Color, and Audio Tape Interface k x * 

V I " - " CMI ~ DATA~ BASE AND CAI , MATERIALS PRODUCTION AND UPDATE* * . u . * 

SUMMARY: Items 3 through 7 require minor modification to software, after clearly defining what must 
be accomplished. Item 8 would be a Category 1 item, already provided, except for the AF- 
wide distribution"— if distribution is not via computer network, then Item 8 is provided 
by the low-cost system. Item 9 requires a degree of interface among AF training programs 
that is outside the scope *of the low-cost^/stem. 



1. On-line capability for non-programmer authors to easily: 

a) Enter, store, and change program Structure, re- 
quirements, and control documents 

b) Enter,"" store, and change program materials 

c) Enter, store and change test items 
ZJ On-line student enrollment/registration 

3. Enter and store program material in an encyclopedic/ 
word order manner 

4. Modularize generic training material coded for recall 
and use In multiple courses with similar requirements 

5. On-line procedures to minimize coordination/ 
approval actions » 

6. On-line entry, storage, and maintenance of related 
training/ personnel records (e.g., AF Form 623 and 
SPOTS Master File) and elimination of hard copy 
"maintenance ' 
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APPENDIX F (CONTINUED) 



FUNCTION 



/ ECI FTD OJT UNT UPT /CATEGORY 



7. On-line entry, storage, and maintenance of 
navigation tasks by weapon system and command 

8. On-line preparation, maintenance and AF-wide 
distribution of refresher and/or upgrade 
training material 

9. Identification of inter- and intra-program 
content overlap and redundancy 



ft 3 
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VII - INFORMATION RETRIEVAL -AND REPORTS 

SUMMARY: Items 3 through 6 could be provided by the low-cost system, at small expense for software. 

modifications. Item 7 is a capability of the low-cost system, but only at the -local level— 
the Interface problems in making this AF-wide would need further definition. Items 8 through 
14 would be difficult to incorporate Into the low-cost system, because they require systems 
at the local level to be aware of, and to act on, data at the AF level. 



1. Courseware evaluation reports 

.2. Individual training profiles, including preservice 
training, education, and occupations, in-service 
training/education, enlistment/classification data, 
and 'quail fication^ertifi cation status . 

3. Courseware development Status, including projected 
completion date and OPR, review completion dateS 
and OPRs, and revision suspense dates, status, and 
OPR * 9 

4. Test Hem analysis reports, referenced to STS, JP6 t> 
and SPOTS 

5. Typing of diplomas, graduation certificates, 
transcripts / • . 

6. f Orf-line entry, storage, and maintenance 6i instruc- 

tor qualifications/status . > 

7. On-line preparation and* distribution -of quarterly 
production Report 
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APPENDIX F (CONCLUDED) 



FUNCTION 



Historical in-service training/education programs data, 

including programs completed » and completion times , by 

student type, command, and location^ y 

•Hi storical i n-servi ce trai ni n$/ educati on ^programs 

data, including Grad Evals by program student type, 

command, and location f 

Historical in-service training/education programs data,' 

including resource (facilities, training devices, 

personnel) utilization by program, command, and 

location A > 

Projected enrollments by program, coronand, location 

»and student types (includes host base ancTpip^ine - 

DDA and ABLE GRAD - students) f 

Existing enrollments by program, command, location > 

and student types 

Standardization and mobility of program and 
.individual training/education data among related 
computerized systems (e.g., ittfCS and APDS) 
Cost analysis data including type of prggram, type/ 
number of student production, resource , utilization, 
command and location * 
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' * * , APPENDIX £ 

• . *• 

LANGUAGE FEATURES REQUIRED BY TOE CAI/CMI* JUNCTIONS 

x .* . Language features were determined by examiningriHe necessary 
CAI/CMI functional capabilities from. Appendix C and determining a 
set of suggested features for accomplishing those capabilities. 
The numbers in. the "FUNCTIONS'* column refer to the Appendix C 
listing of <3U "and CHI functions. The numbers i*h the "LANGUAGE 
FEATURES" column refer -to the listing of features in Section 
5.2.3- of this report. . > 



FUNCTIONS 



LANGUAGE FEATURES 



All 



VA-V€2, VIC3, VIF' 

IA-IVB4, tfl Al- VIC, VIF, 
V1IA1-VIIA4C. VIIB6-VIIB7 

IB-IB6, VA-VIF, VIIA4C," 
VIID-VIIE7. 

IA-ITIC6, VIIA1-VIIA4B, 
VII81-VIIB7 

IB-IB6, VA-VIIA, VIIA4C, 
VIIB3-VIIB5, VIIB8, 
VUD : VIID9v VIIH1-VIIH3 

YA-VC2, VIG3-VIE6 

VIC4-VIIA2A, .VIIB1-VIIB8, 
VIIF-VIIH3 

.VIE1-VIE4 



2, 4, 5, 6, 7', 8, 10, 11, 
12, 13, 14, 15, 17, 20, 21, 
22 

3 " 
9 

16 . 

18 . 

19 , 23 
24 

25 

26 
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1.0 SCOPE 



This specification establishes th? requirements for an operationally 
configured, low cost* computer-roan aged instruction (CMI )/computer- 
assisted instruction (CAI) system. The specification includes those CMI 
and CAI functions identified- as having potential payoff from an oper- 
ational viewpoint^ and excludes "nice to have" or "rich" capabilities for 
support 'of research and development (R&D,) and other non-operational 
functions. 

The system shall be a dedicated loca*! system, rather than a large 
centralv system serving, more than one Technical Training Center (TTC) or 
a system, made up of several s^tand-alone mini -systems.. A modular approach 
to expansion shall be incorporated into the design, to facilitate such 
trade-offs* as fewer, on-line students in exchange for heavier CAI usage, 
and to accommodate increasing memory pr mass storage in order to handle 
increased loacls. 

The system, including the on-line CMI/e&I data base, $rall be sized 
to provide operational CMI /CAI support for at least five hundred .(500) 
students per shift for two shifts. Further design* assumptions are: 
these^ students are distributed equally across five (5) different courses; 
each student averages, five (5) CMI transactions per shift; not more 
than 10% of the students will use CAI at any one time; up to nine (9) 
administrators (instructors or supervisory* personnel) can be on-line 
simultaneously; and batch processing is accomplished off-shift (i.e., 
during periods of low or no student CAI/CMI load), except for essential 
recovery of lessons which are stored off-line. The hardware system shall 
be adequate to provide the functions described in Section 3.0 of this 
specification while meeting the constraints and assumptions described 
above. 
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2.0 APPLICABLE DOCUMENTS 

•The following documents, of , the issue in effect on the date of 
invitation for bids or request for proposals, form a part of this specifi- 
cation to the extent specified, herein. Any reference to the following 
documents, by basic number only, in other paragraphs herein shall be 
deemed to apply to the issue current at the time of issuance. In the 
event of conflict between the documents referenced here arid other detail 
content of Sections- 3.0 and 4.0, the detailed requirements of Sections 
•3.0 and 4.0 shall be applicable.? 

2.1 .GOVERNMENT DOCUMENTS v '.*'.. 

; MILITARY SPECIFICATIONS * • 

MIL-H-46855 Human Engineering Requirements For Military Systems, 
•Equipment and Facilities - 

MIL-E-6051D Electromagnetic Compatibility Requirements, Systems 

MIL-Q-9585A Qualify Program Requirements 

MIL-T-2399J ^ Training Devices, Military: General Specification 
* For 

MILITARY STANDARDS » 

•* MIL-STD-785A Reliability Program For Systems and Equipment 

: . Development and. Production 

MIL-STD-130 Identification Marking of U.S. Military Property 

MIL-STD-143B Standards and' Specifications, Order of Precedenqe 
For The Selection of r * v \ 

MIL-STD-470 Maintainability Program Requirements (For Systems . 
and Equipments) 

MIL-STD-490 Specification Practices 

MIL-STD-882 System Safety Program for Systems/fcnd Associated 
♦ Equipment: Requirements for 

• > MIL-STD-1472A Human Engineering Design Criteria for Military 

Systems, Equipment, and Facilities 



¥ 
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3.0 FUNCTIONAL REQUIREMENTS 

This Section describes the functional requirements for the low cosjt 
CMI/CAI system. - . . **'*;• 

3.1 STUDENT TESTING AND EVALUATION 

The system shall read students' test forms, score tests, provide 
each student with feedback on test results, and record- the results in an . 
on-line data base. The test results shall lie made available in printed 
reports for evaluation and quality control purposes. The system shall 
support at least the following types of tests: aptitude tests, lesson 
and block tests , performance ^checks , and atti tude measures. The types ; 
of allowable questions shall include at least true-false, multiple 
choice, and scalar. , 

The system shall include on-line and "off-line testing* The on-line 
testing shall be supported by interactive terminals. Off-line testing 
shall be supported by management devices, which shall include optical 
mark readers, badge readers, off-line response devices,* or - other 
appropriate-devices to acquire input from students and instructors; 
printers to produce hawi copies of student status reports*; and other 
devices as appropriate. 

3.1.1 Test Scoring and Feedback* Capabi li ties 

" The system shall scofe those tests for which test keys have 
been entered into the data base (see Section 3.6). The results from tests 
shall be available in printed format for both' the student and instructor. 
If the test is a pass/fail test, training managers shall be able to specify 
the pass/fail criteria. The criteria shall include at least the following: 

0 Answers can be scored either right or wrong, and a designated 
percent correct can be required to pass the test. 

0 Objectives can be defined within the test ahd a subset of the ' 
test questions -can be identified as "relating to each specified 
objective. It. shall be possible to use a question for.more than 
one objective. 

0 Any objective can be designated as* a critical objective and the 
test will be failed if the objective is failed. 

0 A subset of objectives c^n be defined* with a minimum number of 
objectives from the subset required to be passed in order to pass 

* A status report shall be printed as part of each student CMI trans- 
- action. Each status report shall include appropriate feedback 
to the student on the lesson or other assignment just completed, 
and shall include appropriate direction as to the student's next 
action— lesson assignment, test assignment, etc. 
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the test. For example, five objectives on a ten-objective test 
can be specified as a subset, and the system can require that 
• -v three or more of these objectives must be passed or the test will 
be failed. \ 

Test keys and other information required to score tests shall be 
entered at an, Interactive terminal. 

* * 

The system shall .score performance checklists completed by -instruc- * 
tors* Instructors shall be»able to certify that a student has either 
passed or failed any of the tests, including performance, checks, from an ' 
Interactive terminal or by entering appropriate information via a manage* - ^ 
ment device. " <x • * 

v 3.1.2 On-Line Testing Capabilities 

The system shall provide the capability for on-line testing 
via an interactive computer terminal. For on-line testing, the order of 
questions and jthe sequence of alternate answers "for each question can be 
scrambled automatically for each test administration. The tfrder of the 
alternatives shall be either fixed or* random as specified by the author*. - . 
The authdr shall be able to make the extent of testing dependent on. the 
student's responses to* earlier questions in the test. A hard copy of the* 
results of on-line tests shall be' available from a management device on 
request. Responses to on-line student test inputs shall be in a real 
time mode. The time lapse from transmission of student input to start 
of displaying feedback shalj not exceed three (3) seconds in more than 
10% of the cases. The time lapse from beginning to completion of the 
response shall, not .average greater than twelve (12) seconds. 

3.1*3 Determination of Pre-Course Student Characteristics 

The system shall provide for storage and retrieval of student 
characteristics data collected before or at the beginning of training, It 
shall §e possible to Use the student characteristics data in examining 
eliminations as a function of aptitude and of years and types of p re- 
service schooling, and in predicting individual rates of progress toward 
course completion. The individual characteristics information shall be • 
stored in the on-line CHI data base and shall be available to the CMI 
system as long as the individual is enrolled in the course. 

The system shall' be capable of storing and retrieving the following 
pre-course data for all students enrolled in courses operating with this 
system: 

' *N0TE: Within the context of this specification, -^author" is used 
| ' as a generic term to describe Air Force personnel— super- 

visors, managers-, administrators, instructors— involved with 
operating this system and, in particular, with entering data x. 
into the systen. % 



Relevant A1r Force personnel data (e.g., ASVAB or AFQT scores, 
* and such biographical data as previous service training and duty 

assignments,); 

."" • Pledge of the course ^terl al based on criterion referenced 
pre- tests or other pre-course measures of knowledge/proficiency 
on training objectives; > » 

0 Learning process skills (ability, attitude, or Interest measures 
such as reading comprehension and study habits); and * x 

ft - * * 

Course-specific critical entry skills (aptitudes, abilities, and 
intereststiwft are specifically related to success in a particular 
traiuiftg**spsc1alty, such as mathematical ability or interest in 
the career field). 

3.1.4 Determi nat j on of Wi thi n-Course Studen t Atti tudes and . 
Interests 

• The system shall make provision for -tests to measure the 
atti tudes .and interests of students. Pre-course, wi thin-couple., and post- 
course tests of attitudes and interests shall be accommodated. The 
results of the teste shalj be stored in the on-line CMI data, base for - % 
use in determining training assignments,,, evaluating instructional'* 
materials and procedures, predicting completion times and rates of pro- 
gress, and predicting performance, as specified by training managers, 

3.2 ^STUDENT ASSIGNMENT > ' 

i 

The system ?hall assign students individually to lessons,, tests, and 
other instructional" alternatives. The system shall be structured to 
allow the training manager to specify what combination of data on lesson 
characteristic^, resource availability, student characteristics, and 
student performance will be used in determining the assignments* of 

individual students to training alternatives. 

• / 

A student shall recefcve the next individual assignment immediately 
after completing the preceding lesson. Training resources shall be 
scheduled *by the computer system to attempt to maximize throughput. The 
computer /Shall track student progress through the lessons and al ternati ves 
in the Various course versions* The system shall provide reports that.*can 
as desired; replace manual tracking of student progress by instructors." 

3.2.1 Assignment to Alternative Course Versions 

1 The system shall support course versions which differ as to 

content (e.g., for different shredouts of a specialty, or for testing and. 
implementing new ^training materials). A student enrolled in a particular 
course version sh&ll be' assigned only the training materials for tJn^T 
version. The form or procedure used to register a student in a course 
shall provide for identifying the student's course and version. s 
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3.2.2 Determination of the Order of. Assignments 

The system shall have* the capability to determine and assign 
the next lesson alternative for an individual student in all modes 
except block remediation. The instructor shall not be required to 
manually tra^ck student progress, or to, determine assignments to train- 
ing alternatives, and shall have capability to perform manual override 
of any assignment. Based on data in the CMI/CAI data base and subject 
to options selected by authors or data base managers, the system shall 
be capable of using at least the foil owing 'information in determining 

a, trainee's next assignment*., v .*••*. 

** . * 

o * 
The assignments complete?* by the trainee to date; 

♦ The performance of the trainee on previous assignments (times and 
test results on previous assignments); 

> 0 The availability of\.the resources required for. existing and 
allowable lesson alternatives; and 

0 The current distribution of students, the numbers of^tudents 
required for any team tasks, and how best to assemble \teams of 
trainees for team tasks. 

3.2.3 Assignment to Alternatives Within Lessons* v 

The system shall make individual student assignments to 
alternatives within a lesson. These can include alternate media (e.g., 
printed vs. audio-visual)*r different instructional approaches (e.g., 
easy vs. difficult)* me Assignments can be based on at >east the 
following considerations: 

0 Availability of the particular media or lesion; 

0 Prior test scores , times* to-complete, or other wi thin-course data; 

0 The student's pre-course characteristics (e.g., general aptitudes, 
abilities, attitudes, interests, or other pre-course data); 

0 Simple*" if. ..then" logical rules which may be used* to select 
alternatives; ami , 

0 Predicted student performance. 

Student character! sties and past performance can be used to predict 
student performance. The* student shall be assigned to the media or 
lesson predicted to result in the best student performance. The system 
shall accommodate at least two assignment rules for each lesson— best 
predicted time and best predicted score— which can be specified at the 
lesson level by an author. If the score rule i% selected, the student 
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shall be assigned to the treatment with the Highest predicted score. 
If the time rule is used, the system shall assign the student to the 
treatment with the shortest predicted time. If the time rule is 
specified and the student is predicted to fail all of the treatments, 
the system will use the score rule. These rules will be used only for 
a student's* first attempt on a lesson. Subsequent assignments shall 
be made using remediation rules (Section 3.2.4;}. Additionally, resource 
availability and system throughput constraints shall override the time 
and the score rules. • N 

The sys,tem shall support rdfdom assignment of a specifiable per- - 
ceritage of the students to the treatments within a lesson. If 15$ 
random assignment Is specified* then a randomly selected 15% of the ^ 
students shall be assigned randomly to the available instructional 
alternatives for that lesson. The random « assignment function will be 
used in collecting data to establish assignment ru.les and to verify 
the effectiveness of such rules. , 

3.2.4_ Assignment to Alternative Remediation Activities 

The system shall make assignments to the available alternate, 
remediation ^materials and media. At least three categories of 
remediation shall be supported: 

\ 

0 Lesson Remediation - The system shall support alternatives for 
use in remediation only. The system shall assign a student to 
the "best" available remediation alternative for second and sub- 
sequent lesson, attempts, with "best" defined to fre the one 
alternative chosen from available alternatives based on the con- 
siderations listed in Section "3.2.3. ♦ 

Lesson- Group Remediation.,- The %/stem shall assign the best 
available remediation alternatives within a related group of 
lessons. 

. 0 Block or Course Level Remediation - The system shall assign tfie 
best available remediation alternatives for the objectives that 
the student is having difficulty mastering, within the block or 
course. 

The alternate remedial selections shall, be accomplished by the 
computer. It shall be possible for instructors to override system 
decisions via management devices and from interactive terminals. 



3.2.5 Test Selection and Assignment 

The system shall provide the capability of assigning students 
to tests. . For each student at each testing point, the system shall 
assign. an on-line (CAI) or an off-line (paper and pencil or performance) 
test, as "available and appropriate. If more than one version of a ^est 




is available, the alternates shall be assigned randomly. Any versions 
already attempted. by the student shall be excluded on subsequent 
attempts. Capability shall be provided for manual override of any test 
assignment by an instructor. On-Hne programs* shall prafide the 
necessary data entry capabilities for implementing £his function. 

• 3*.2.6 Assignment to Supplementary Skill Training at Course Entry 

The system shall be capable of identifying those students 
•with deficiencies in critical entry skills or in learning skill areas 
and of assigning students to supplementary or remedial skill training. 
The system shall identify such students based on critiera established 
by course managers for performance on aptitude, ability, interest, or 
other precourse or within course measures. Supplementary/ remedial train- 
ing lessons shall subsequently be assigned to students in accordance •' 
with hierarchy and resource availability, constraints established inHhe 
QMI/CAI data base by course managers. 

3.2.7 Additional Resource Management Considerations 

Based on information entered into the CAI/CMI data base, the 
system shall track which resources are required for each lesson, if they 
are available, and where they are located. Managed resources can include 
classroom spaces, lesson materials, media devices, work stations, 
simulators, and other training equipment. The system shall be able to 
assign and direct students to training resources outside of their assigned 
learning centers, to allow effective sharing of resources by students 
in several learning centers. * 

The system shall assign students in a manner that will (1) tend to 
distribute students evenly throughout the course, (2) attempt to keep 
as many assignment options as possible open for the students, and (3) 
make efficient use of avaflable resources. Students shall be assigned 
to the learning centers with the highest percentages of available space, 
and shall be assigned to blocks in such a manner as to opt i mi ?e' the ▼ 
distribution of students across learning centers. In the process, the 
system shall consider team assembly, desired loadings within blocks, and 
the capacities of learning centers. These considerations shall not, 
however, override the hierarchical constraints or priority given to 
each block. - 

The* computer system shall base the above management decisions on data 
entered into the on-line data base. Once a i)urse version is operative, 
training management personnel shall be required to enter only courie 
changes as they occur. These include changes in the course struct%*e 
(e.g., lessons added or removed, or the prerequisite order of assign- 
ment changed), and resource availability (e.g., new equipment added, 
and equipment removed, inoperative or in maintenance}. The changestshall 
be "entered using on-line programs and shall take effect when entered. 
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3.3 S7U3ENT PROGRESS MANAGEMENT 

The system shall support management of a trainee's progress through 
a self-paced individualized training course. These capabilities shall 
range from lesson scheduling to progress prediction and management. 
Both the student and* the instructor can be informed, at any time, as to 
the student's progress-- ahead of,** behind, or on schedule. 

3.3.1 Scheduling Student Entries Into The Course 

At the end of each training day, theirs tern shall identify 
the number of vacancies in each course operating under the* system. This 
information will allow the student squadron or othfer administrative 
authority to effectively plan and program course entries and minimize 
student delay awaiting training. \ 

3.3.2 Graduation Forecasting * * \ *' 

The system shall, as part of the student monitoring and 
management process* (Section 3.3.3), predict the date onVnicto a student 
will complete the course. Learning center rosters shallXbe available 
on request, and shall identify students and their predicted graduation 
dates during the datse/time interval specified in the training! managers 
request. The rosters shall be available in printed form, and \f or display 
at an ihteracti ve* terminal . The" average response time, from swbmisarton 
of a request to completing printout of a roster, shall not exceed two 
(2) seconds per student. The system shall also, on request, generate , 
graduation reports for the local training center personnel office. 
These reports shall be generated on the central sHe line printer. 

9 

« The system shall be configured to facilitate* interface directly with 
the training center personnel management computer system. To implement 
this function, the Air Force wilT specify the format of the data and ; 
the medium in which the training tenter personnel computer wi-11 receive 
the required .information. Any required* changes to the training center 
personnel computer software will be accomplished by Air Force personnel. 

3.3.3 Monitoring and Management. of Student Progress to a Target 

The system shall determine a target progress rate f6r each 
student. The rate for»a student shall be used to generate a targeted . 
time for completion of the 'course and for completion of individual 
blocks. The target progress rate for a student shall be* based on pre- 
course data and on available wi thin-course performance data. The targets 
-shall be relative to average. completion times -for blocks and course. A 
student's target completion date shall be calculated based on the target 
progress rate, expected swft length, predicted days the student will 
be in class, the student's performance record, and the percentage of the 
course remaining. Each student shall have access to his/her target time 
to complete, for each block* Both the student and the instructor shall 
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receive feedback en the student's actual progress rate relative to 
targeted progress rate. The student shall be given inf pronation, on 
the first status report J of each training day,, that wilt let the Student 
determine how actual progress to date compares with' predicted progress, 

3.4 SUPPORT "FOR *SUIDANCE AND COUNSELING /, . 

• ... , > ■* 

t The system shall provide computer generated reports to support - 
student" guidance and counseling by instructors and course management 
personnel. If the necessary tests are included in the p re-course 

.assessment,' the reports shall identify entry skill deficiencies as well 
as deficiencies in wi thin-course performance, the instructors will 
determine which students are to receive counseling, and can retrieve , 
from the computer both the .background characteristics and wi thin-course * 
performance information needed to support their guidance and counseling 
activities. "•',*" *• • , i * 4 

3.4.1 Identification of Students Needing Special Attention^ 
Course Entry 

The system shall identify students predicted to be proficient 
or deficient with respect Jto mastery of course*or training objectives 
specified by course personnel. *This identi f1 cation ahd reporting i s 
initiated at the beginning of the training process to focus instructor/., 
management attention toward students most in need of special, training 
procedures, guidance, and counseling. , . g * 

The identification, process shall produce computer generated lists 
of those students measured as being either proficient or deficient on 
the basis of their pre-course assessments. This initial listing shall 
report, by student, individual scores oh the pre-course critical entry 
skill variables (e.g., reading, math, or study skills). A second 
report shall provide individual scores for all students on those 
variables identified as critical entry sKills, and^shall also flag 
deficient students and their, related scores. These reports shall be 
available for display at the interactive terminals. • -r?*^ I 

3.4.2 Identification of Marginal Student Performance 

i 

The system shall produce reports identifying and reporting 
students predicted to have difficulty mastering further training 
objectives based on recorded wi thin-course performance. The reports 
shall be available on denand and utilized to direct timely remedial/ • * 
counseling assistance to students most in need. The system shall, be 
capable of assigning those students identified as marginal to 
specialized remedial training (e.g., to special instructor tutorials, 
remediation sessions, or special skill training for correcting learning 
or study ski 11 weaknesses). 

The system-generated reports shall include a list of those students 



net meeting predetermine? Hmits of satisfactory predicted progress ' * 
rates, or scoring below the training manager 1 s* criterion ievels for 
training objectives. They, shall include actual time and score data. 
The students below the course-established r m1Vti mums for -acceptable per- 
formance will be flawed as^possi^le candidates for elimination or , 
other established administrative action.- ' . 

3.5 .COyPUTER-flDED'lNSTRUCTION (CAI) . 

Lesson materials c§m be entered into, stored in, and retrieved and 
delivered by the computer system. When assigned by the system and • 
requested by the student, they can be displayed at an interactive terminal 
The lesson materials can" include text, , questions, graphics, ano\ special 
v purpose .programs. Student interactions can range frpm a simple "turning . 
the pages" type of prograln .through very sophisticated interactive 
exchanges via special purpose programs . Student ffrtferacjfclo n s aro— ~r — - — 
principally v|a a. terminal ^eyboafdjNld, if appropriate, Via a touch 
panel, light pen, or other device*.^ * . • 

. ' 3*5.1 CAI Applications 

The* CAI provided by this system shall support at. least the 
following! fiveijCAI |unctions for use in Air Force, resident technical 
training: V 

% " ' .< • 

* 0 Dri)l and Practice, with cbrrettion and guidance, of basic skills 
_ : and knowledges learned via other media; 

: , 0 Tutdrial— use v of CAI for lessons which are particularly difficult; 

°. Simulation— use' of soeclal purpose -programs to simulate equipment 
operation or maintenance. processes; f 

0 Review and Remediation-ruse of^CAI to quickly review the content , 
of a block prior to a block test or to remediate failed objectives 
following a block test failure; and •* . . " 

*•.-•' 0 Study Skim— use of CAI to teach specific study skills to students 
Who .are deficient in these areas. . 

- 3.5.2 CAI "Capabilities . 

A CAI lesson "Shall consist of *a series of frames', or equiv- 
alent means of,storiag and* presenting information. The system shall 
support at least the following types of frames, or. their equivalents: 
- text frames that can .contain at-least *4 pages of information (a page can 
contain at least El lines); question frames ran* documentation frames 
which the student cannot access.. The .documentation frames are reserved 
for course authors to>ecord development status data. The system shall 
provide f or at least 100 objectives>er lesson and VO0 frames per . 



objective. 



The system shall support CAI lessons of varying levels of sophis- 
tication. The basic CAI lesson «wi 11 consist of text presentation, and 
will allow response to multiple-choice and true- false questions via a . 
typewriter-like keyboard J The system shall be able to provide feedback , 
messages spec/if to* to the! ndi vidua! student's responses and number of , 
attempts on the current jpjective. The system shall also allow branching . 
to any available frame Jwressing the current objective, or to the first 
frame of any other objective within the lesson. The lesson progression 
shall be adaptable to #ie student's responses. The branching selection 
shall allow branching to any available frame addressing the current 
objective pr to the first frame of any ottaer objective within the* lesson. 

• ' The lesson progression shall beadaptable to the student's responses. 

The branching selection shall be based on the student's prior responses/ 
p e r fo rma nce w ithin - " the le sso n. : ~ — — — 

* * » 

For more' sophisticated CAJ lessons; the system shall support 

* graphics presentations^ col or displays, student entry of constructed 
responses (open-ended- English answers), contrdT of supplementary slide 
or filmstrip presentations, control of audio tapes ,*and adaptations 
based on the CMP on-line data base (e.g., ASVAB or AFQT scores, P re- 
course assessments, and within-coarse performance-data). The system, ^ 
shall support the audio tape player and the projector via an external 
computer-controlled jack on the interactive terminal. To facilitate 
student interactions in these CAI lessons, the system shall support the fc , 
usual keyboard inputs and shall also support an additional device^such 

as a light pen or touch panel. To facilitate preparation of graphic 
displays, the system shall support a digitizing tablet- or equivalent 

device; » v 

• -. • • 

The system shall have the capacity for on-line mass storage of a 
minimum of 150 lessons, available for immediate access. The lessons 
are assumed to average three objectives per lesson', 60 frames (or 
equivalent) per objective, 2 pages per frame, and .15 lines per page . 
(average line length, 6CT characters). Lessons in dally use shall be 
available on the system at.all times, with no more than a ten (10) second 
.- . delay in initiating student access. When a student has indicated com- ^ 
ple'tion of a frame, the average time .for the system to determine and 
initiate the ncjxt display shall not exceed three (3) seconds. The 
- average time to complete the display shall not exceed twelve \U) 
seconds. * 

V inactive lessons whi^h are stored off-line shall be recoverable for 
formal on-line accessibility not later than the start of the next training 
day following the request for restoration. . 

3.5.3 On-line Learning Process/Specialized Skill Training , 

The CAI system shall support presentation, to either students 



or instructors, of specialized training in interactive formats. This 
training may include, for .example, materials designed to teach study 
habits and skills, test .taking skills, and memorization skills to * 
'.students, or to teach diagnostic, tutorial, authoring or counseling 
skills to instructors, , - 

i 

3.6 CHI DATA BASE AND CAI MATERIALS PRODUCTION AND MAINTENANCE 

■ •< .'* 
To effectively perform. CAI/CMI training functions, the system 4 

.must be informed as to the trainees (name, social security number, 
course, etc.), the course structure (tests, lessons, resources, allow- 
able sequences of lessons, etc.)* and student performance (lessons , 
completed, test scores within the current block, block .completion times, 
etc.). In the case of GAI, all of the' lesson information (text,*, 
questions, graphics, branching instructions, etc.) must be stored with- 
in the compute^.y stern. The following paragraphs describe the 
requirements for entering, storing, and maintaining these types of 
information. « 

3:6.1 CM* Course Definition Data Base \ , > . . ' 

The systerri shall provide the capabilities'whicKwill al!6w. 
training managers to enter the course structure into the "system. The 
program(s) shall run.on-1ine--i ,e» » a training manager shall be able to 
use an interactive terminal % £pMirectly enter, store, display, and Change 
course related, information. " To facilitate use, the on-line system shall 
-use graphics to display the course structure (i.e., the allowable paths 
through the lesson materials)', and there shall be an on-line, capability 
to access. each file in the CAf/CMJ 3ata base. Additionally, it shall not 
be necessary for-. authors to be familiar wi th any programming language 
in order to* use these capabilities effectively. 

* 3.6.2 vAutomattc Validaliofrof Course ^ata Base Integrity 

$ 

. V Before a- new course or course version is implemented, 
necessary information/description relative to all of the blocks, lessons, 
courseware, and tests in the course musV be entered into the data base. 
The system shall include one or more computer programs for determining 
•if all required data base records defining the* course are present and 
cons-i stent. The program(s) shall be-designed to facilitate use by 
authors and shall not require knowledge* of a programming language. To 
the greatest, extent possible, the program(s-) shall update the on-line 
data v base, if. this capability 1s enabled by the training manager. The 
program(s) shall run in not more than fifteen (15) minutes in validating 
the data base for a course. * * * 

- . 3.6.3 Production of .On-Line Tests" . * 

* The CAI system shall, include the capability for on-line test- 
ing. This shall be implemented via* an. on-line test item editor, or 




equivalent capability, for entering test questions, their alternatives, 
and the necessary control information. This capability shall. allow the 
author to enter test items by typing text at an interactive terminal,, 
arid shall also format the questions for the author and allow the author 
to decide whether or not to (a) randomize the presentation order of 
the test items and the alternatives within test items, (b) control the 
number of attempts the student can make on each item, and (c) enter . 
student feedback messages to follow either correct or incorrect answers. 

•3.6.4 CAI Authoring • . s 

The system shall provide an "on-line CAI authoring editor, oV 
equivalent capability, which shall assist in Structuring the- authoring : 
tasks. 'This capability, shall provide standardized formats for* text and 
question frameR and shall provide the capability to copy or share text 
frames, questions, and graphics between lessons. A users manual strati ~~ 
be provided for the CAJ production process. Additional instruction shall 
be displayable during the authoring process via a HELP' request. The 
HELP information shall include operating instructions for the program, 
plus guidelines for preferred Instructional practices. This capability 
shall be simple enough that an author without prior programming 
experience can learn to use it in not\more than two weeks. An, author 
shall have access to any frame in any "lesson that is allowable within # 
system security constraints.- Students shall not be able to access -\. 
lessons in the author mode. Additionally, the program(s) shall provide . 
capability to readily change any of the course materials stored in the.v 
on-line frata base. • • .• 

3.6.5 Production of Off-Line Materials 

' It shairbe possible to use the system in producing tests 
and instructional material s-which are intended for Use off-line (e.g., 
programmed texts). Programs shall be designed for use in writing, * 
reviewing, and revising the tests and instructional materials and for 
printing the final product in a format ready for reproduction. It shall 
be possible to print a limited* number of copies in order to facilitate 
early reviews and small group tryouts. This system capability shall 
also facilitate conductinq snfall group- tryouts 'on-line, and sha.ll be 
useable for acquiring information on the students' reactions to in- 
structions 1 materials. 

3.6.6. Production and Maintenance- of Course Documents 

• * • ' » • i - ' 

An interactive capability stmilar to that provided fo/.test 
items and CAI authoring shall be provided for developing, storing,* main- , 
taining, and retrieving course documentation such as Plans Of Instruction 
(POIs), course charts, and lesson outlines. These documents shall be 
available at any time for review or revision, and limited numbers of, ^ 
copies"' can be printed as needed. , ' * 
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3.7 INFORMATION RETRIEVAL AND REPORTS 



Information relative to the trainees, courses, course materials, 
tests, test items, and other components of the system shall be stored, 
and available, for periodic (hourly, daily, weekly, monthly...) analyses . 
and reports. These functions are intended principally ,to support in- 
structors, managers, and administrators. ,r They can be valuable to in- 
structors involved with course management and student guidance. They 
can also be valuable in providing an effective quality control mechanisms 
for improving the .management and administration of training, and 
evaluating and' improving the process of training program development and 
implementation. A report requested at a terminal shall be provided to 
the, requester only if security-requirements for access .to the requested 
information are satisfied. v ^^~- 

.3*7.1 Reports for Instructors 



Data shall be collected, stored, and made available* in 
reports structured to meet instructor needs. The reports shall include 
■but not necessarily be limited to the following: student rosters, * 
individual block progress reports, individual biographical reports, and 
absence summaries Jor learning centers. They shall be available via 
either, the interactive terminals' or in printed form. The instructor 
shall also have access via the terminals to all other student data not 
included in the above routine reports',* but subject to established security 
constraints. .. . • # 

'3.7.1,1 Learning Center or Classroom Rosters. The system 
shall provide, .on request, via either an interactive terminal or in 
printed form, at least four types of rosters: resource, assignment, time 
management, and homework. The rosters shall include social security- . 
numbers and names for the students included on the list. The resource 
roster will list those students in the learning center who have resources 
assigned to them, the types of resources, ano> the learning centers to 
Which the resources are assigned. The assignment roster will list the 
current training activity assignment for each student by learning center. 
The time management roster will include carrel (or position) number, 
'absence status, current block, and rate of progress for individual students 
by learning center. The homework rostef will identify the amount of the 
course completed to date and the proportion completed through voluntary 
homework for individual* students by learning center. After the terminal 
has transmitted a roster request to the central site computer, the * 
average processing time per student shall not exceed one (1) second. 

» 

* 3.7.1.2 Individual Block Progress Reports. The individual 

block progress report will summarize student progress for the block 
specified. The information in the block progress report shall 'include 
at least the following: 

o • 
Lessons completed in the block* • ** 



0 Numbers of attempts, and scores on each attempt, for each lesson 
and test; ... - 

0 Objectives-failed on each tes^t; and * i 

' ' 0 Time spent on each attempt and total time spent in thejrtocjtr 

3.7.1.3 Individual Student History Reports. The individual 
student history report will include the student's scores on pre-course 
aptitude and interest measures, scores on any critical. entry skill tests, 
time spent in remediation following a block failure, arfd any lesson 
objectives that were failed more than twice. 

3.7.1.4 Absence Summary. * The absence summary shall be ,a 
block-by-block summary of the absences reported to the system and shall 
he r,apah.l£.JilJ^ indicate _ 
when the student was reported absent, when* the student returneavnpr 
iQng the abiences lasted, and the reasons for the.absences. • 

3.7.1.5 On-Line Display of Student DataV Capabilities shall 
be provided for retrieval and display of available student information in 
response to requests which meet security requirements*. 

3.7.2 Training Management Reports 

The following information, intended for use by training 
managers (shift supervisors and above), shall be provided in the form 
of standard" reports available on request to authorized users: 

0 Students awaiting training, by course; - 

0 Numbers of students in training by course, learning center, and 
» Technical Training Center; 

0 Distributions of completion times for individual blocks; 
■ 0 Distributions of course completion times; r 

0 Differences in student performance between parallel learning 
centers*, or Technical Training Centers; 

0 Instructional resource (e.g., facilities and training devices) 
utilization; 

0 Utilization and qualifications of instructors; 

* Learning centers are "parallel" if the same course content is 

taught in the centers, and are not parallel if the centers teach 
' different content. * . • . ^ 
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0 Summary reports of studen$ completion times and final grades, by 
block and course; and 



Summary reports of disenrollment and elimination data. 

3.7,3 Course Evaluation|Summary 

t The system shall include a Course Evaluation Summary (CES) 

which summarizes student performance on individual lessons within blocks 
and for entire blocks. It is intended for use in monitoring and 
evaluating small -group tryouts of new materials as well as student per- 
formance on established materials. The CES shall,. contain at least the 
following items of information: 

0 Fi rst^attempt lesson and block test socres; 

0 Final lesson and block test scores, 

0 Means and standard deviations of first-attempt lesson and block 
test scores; » • 

0 Means and standard deviations of final lesson and block test 
scores; - • . 

0 Means and standard deviations of first-attempt lesson and.bljbck 
times; * i 

■ Means and standard deviations of lesson and block completion 
times; 

Separate score and time data for alternative modules within 
J lessons; 

; Score data by objective within, lessons; 

0 First-attempt lesson failure rates; 

0 First-attempt objective failure rates; 

0 Correlations indicating the relationship between predicted and 
actual lesson and block times and scores; and 

0 Summarized data on lessons, modules and objectives with first- 
attempt failure icates above a specifiable percentage. 

. The CES shall be a batch mode report. * An on-.line program shall be 
provided, however, for submitting requests for the CES for off -shift 
processing. 
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3.7.4 Test Item Evaluation Report 

The system shall provide a Test Item Evaluation (TIE) report 
that provides detailed information on the characteristics of block and 
lesson tests. The TIE is intended to support evaluations of test 
reliability and validity. The TIE shall included least the following 
items of information: 

. 0 The numbers (or percentages) of students selecting each alternative 
answer to test questions; 

0 Percentages of students answering questions correctly; 

0 Average test score of students answering each question correctly; 

0 Flagging test questions missed by over 70 percent or by none of 
the students; 

0 I tern- remainder correlations for each question; 

0 Means and standard- deviations of scores for each objective and for 
each full test; 

0 Distribution of scores on the full test; 

0 Alpha reli ability coefficients for each objective and for the_ 
full test; and 

0 Crjteri on- referenced reliability coefficients for each objective 
and for the full test. 

The TIE shall be a batch mode report. An on-line program shall be 
provided to" submit the report requests for off-shift processing. 

3.7.5 CAI Evaluation Data 

Detailed quality control and evaluation data shall be 
collected by the CAI system. This information shall include at least 
the following items: 

0 Percentages of students feel ecting each alternative on each . 
~ question, by attempt number (first attempt, second attempt, etc.H 

0 The mearfs and standard deviations of times to respond to questions 
and times to read text passages; . , 

0 Lists of unanticipated responses to constructed response 
* questions; ^ 

0 Lis.ts of open-ended student comments regarding the CAI lessons; ^ 
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»- Detailed records of students' pathi through CAI lessons; and 

8 o ) I 

Summary statistics on major units fof CAI lessons, such as times 
to complete and cumulative scores.^* 

]• This information from CAI lessons shajl be available to authorized 

requesters in batch mode reports. An on- 1 fine program shall be provided 
to submit requests for off-shift processing of these reports. 

3.7.6 Course Development Management tteports 

. The program or programs provided for use in developing on- 

line and off-line tests and materials shall also capture and store in- 
formation regarding an author's work using the editors, and the status 
of the associated course material development effort. This information 
shall be summarized in management reports for monitoring courseware 
development, production, review, revision, and implementation. The 
reports shall include such information as identification- of the author 
responsible for developing, reviewing or revising a lesson, how much time 
has been expended on a particular task, how much of the task has been 
completed, and the expected (i.e., scheduled) completion date. 

3.7.7 Performance Forecast Reporting 

The system shall include a report which describes a student's 
performance levels in completed technical training. This information 
is intended principally for a student's supervisor in a post- training 
duty assignment, and. for Field Training Detachment (FTD) or On-the-Job- 1 
Trailing (OJT) managers who need reports of individual student perfor- 1 
mances (e.g. ^-scores and times to complete) in resident technical train- 
ing or other in-servicie training. 

-3.7.8 Special. Purpose Reports ^ 

Because standard reports will not always satisfy all manage- 
ment and administration requirements, the system'sball include an open- 
ended information retrieval system that can be used to answer ad hoc 
queries regarding students,. materials, and training effectiveness. The 
user shall be able to specify the variables which he wishes to examine', 
e.g., types of students or courses by specified time periods. The 
output shall consis/t of the number of pertinent cases identified and the 
mean, standard tie vi^ati on, and range for each included variable. This 
f basic data .re tr 5 ie vail program shall also be interfaced to one of fhe - 

standard packages of statistical programs such as the Statistical Package 
For The Social Sciences (SPSS) or the BMD Biomedical Computer Programs, 
to permit extensive analysis of student data. This data retrieval/ * 
analysis capability shall be a batch mode function. AnjwT^Tine program 
cha11 Ko ^^.iA^A however, for submitting requests for off-shift pro- 



shall be provided, 
cessing. 
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4.0 HARDWARE REQUIREMENTS 



A computer hardware system shall be provided to support the instruc- 
tional, administrative, and management functions required by Section 3.0 
of this specification. The system shall consist of commercially avail- 
able equipment with the capability for handling ttie types and numbers 
of jobs specified in Section 3.0. The hardware components provided for 

* this system shall meet the general intent of Military Specification 
111^-46855, Military Standard MIL-STD-785A, and of the other documents 
listed in Section 2.8. 

"The computer system shall provide a capability for .modular buildup 
' to the terminal configuration necessary to meet the functional require- 
ments and assumptions in Section 3.0 of -this specification. The system 
shall operate on-ljne for two eight-hour shifts, five days per week. 
Batch operations will be carried out during the remaining hours or 
during weekends. „ ' 1 ,™ — ,', , f * „ 

4fl MAIMFRAME ANP PERIPHERALS 

The central computer system (i.e., mainframe and peripherals in- 
cluding any communications network processing unit) shall be .clustered 
and housed in a room as centrally located among classroom buildings as 
is practical.- This room shall be 4 central- point for printing CHI 
records and administrative reports and for performing software main- 
tenance, t 

The centfral system shall represent the minimum configuration con- 
sistent with meeting €he requirements ;of this specification. High speed 
communication between two or more processors of the sar.&jmake and model 
shall be possible, to allow for" future expansions that mrght require 
.flSTHIple central processors. Consideration shall be given to selecting 
a computer that does not require such special environmental conditions^ 
as chilleti water and humdification. " Sufficient air conditioning and a "\ 
raised floor shall be installed with the system. 

4.1.1 Central Processor 

I « ^ 

The, processor shall be^ capable of simultaneously supporting 

at least the number of student, management, and administrative terminals 
-or other devices needed to meet the functional requirements of Section • 

3.0 above. To support routine software operations, simple on-line source 
/ program editing operations such as line insertions, deletions, and 
r replacements shall be accomplished in an average response time of not \i 
i~mdre than oue (1 ) second, and more complex on-line operations such as $ 
j file I/O shall be accomplished in an average response time of not more 

* than three (*3) seconds. 

! An instruction set supporting at least integer and floating point # 
arithmetic, string, character address*! ng, branching, logical, looping, 



and assignment operations shall be provided. Instructions- or groups 
of instructions shall also be available to implement array subscripting 
and record addressing. ^ 

At least the following peripheral devices shaTl be attachable to 
the central processor: high speed mass storage devices, off-line storage 
devices (e.g., tape drives), line printers, and an operator console. It 
shall be possible to add central memory expansion units to the processor 
without degrading the services offeree by the operational system. 

A capability for validating data following transfers shall exist, 
to, insure that transfers are successfully and correctly accomplished. 

\ The processor shall be capable of supporting batch processing, but 
not necessarily while supporting on-line CAI and CMI operation. 



4.1.2 Central Memory 

Sufficient central memory shall be provided to support the 
terminals and produce the response times dictated by the functional 
requirements of Section 3.0 above and by Section 4.1.1. Any type of 
central memory (e.g. , core, MOS, bipolar) .shall be acceptable, provided 
the required response times and validations requirements are ipatisfied. 

4.1.3 Mass Storage 

1 

Mass. storage shall be provided to support the instructional 
programs and data required by Section 3.0, and the system programs and 
data. There shall be at least two mass storage devices, to provide backup 
in the event one device malfunctions, the mass storage transfer rate 
plus overhead shall be sufficient to accomplish the required response 
times. 

Channels attached tto the mass storage devices shall be capable of 
accomplishing input/output (I/O) transfers at sufficient rates to fulfill 
the response time requirements of Sections 3.0 and 4.1.1. 

4.1.4 Off-line Storage 

» 

Off-line storage shall be provided to support the functional 
requirements of Section 3.0. For backup and reliability purposes,' this 
storage shall be accessible via at least two off-line storage devices. 

The eff-line units shall be accessible for backing up information 
from on-line storage devices and for recovering information which has been 
temporarily stored off-line to lessen the on-line mass* storage require- 
ments. Off-line storage shall be sufficiently accessible to fulfill the 
requirements for recovering archived CAI lessons (Section 3.5.2 above). 
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4.1.5 Central Site Printing 

Capability shall be provided for printing such information 
as class.rooni rosters, course evaluation summaries, te>( Item evaluation 
summaries, selected student information, CAI reports, on-line and.off-. 
line course materials, software programs, and computer maintenance 
Information.' One or more central site line printers capable of printing 
the 128-cbaracter ASCII set shall be provided. With an even distribu- 
tion of all characters, a minimum rate of 500 lines/minute (128 € 
characters per line) is required. 

4.1.6 Computer Monitoring 

A method and equipment shall be provided' to furnish computer 
^andjperipheral device Information, initiate diagnostics, and 
generally monitor and control the operation of the entire system^' v 

4.2 COMMUNICATION SYSTEM 




ERIC 



Conraunfcation hardware and software sufficient to allow remote or 
local operation of the terminal system specified in Section 4.3 below . 
shall be provided. Standard protocols, electrical interfaces, and data 
rates shall be used throughout. This communication system sha'll be 
capable of supporting the mix of terminal capabilities specified in 
Section 4.3 and the response requirements of Section 3.0 of this 
specification. The communication system shall fce moduterly. expandable 
beyond the minimum number of terminals required in this specification • 
- even though the software and hardware system is initially configured,^ 
for that number of terminals. m • . 

\ 

4.2.T Computer/Network Interface . 

The network processing hardware at the computer site shall 
be able to transmit data to and receive data from the total number of 
terminals required by this specification, and the electronics shall be 
mosfularljpexpandable. to handle additional terminals. 

The computer electronic ports for system terminals shall meet, 
industry-wide standard interfacing requirements. The network processing 
equipment shall have the capability of detecting and correcting trans- 
fer errors on characters and control codes received from the terminals. 

For transmitting data between the computer and terminals at 
locations remote from the -central site, the most cost effective method 
shall be provided. 

4.2.2 Network • /- 

* The network shall consist of dedicated voice grade lines, 
configured to provide the data rates needed to satisfy the functional ( 

* • .** 
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requirements of this specification. 

4.3 j TERMINAL SYSTEM - ^ 1 v 

*' •* V 
The terminal system shall be composed of a mix of types of 
terminals and other devices that Includes: 

(1) alphanumeric, graphics, and color display types of terminals 
for students and administrative use; and 

(2) management I/O devices to provide the required CMI functional 
capabilities, / 

This compatible set of devices shall be located in or near, class- 
rooms, and shall be capable of remote operation with the central 
computer. , 

* » 

4.3.1 General Characteristics 

, All display terminals selected shall have the electronic, ' 
display,, and keyboard characteristics as listed in Sections 4.3.1.1, 
4.3.1.2., and 4. 3. 1.3 feel ow. Management devices shall conform to Section 
4.3.1.K 

• - * 

4.3.1.1 Electronic Characteristics. Each terminal shall be 
capable of sending and receiving data at selectable industry-standard 
rates* Transmission shall be via an industry-standard interface. 
Terminals or associated communications equipment shall have at least 
the capability of detecting validation errors on single incoming alpha-^ 
numeric and control characters, with all control codes and character 
codes conforming to industry standards. Any terminal g hall be ■ 
electronically interchangeable, with any other at any communications 
port. ,> . " » 

4.3.1.2 Display Characteristics. Each display terminal 
shall provide at least 24 lines of display, with 64 characters or more 
per line, on a screen of at least 12 inches diagonal measurement. The 
96 character ASCII set, consisting of upper and lower case alphabet, 
numbers, and -punctuation shall be available. The cursor shall b^ 
controllable so that it can be located to any position" on the ^reen. 
Bulk erase capability stjall be provided. * 

* if * WP* * 

4.3.1.3 Keyboard "Characteristics. The keyboard Jon each 
display terminal sha\l be in the standard QWERTY format for alphabetic 
characters,* with numbers and ^punctuation keys also/available. All keys' 
must be plainly and permanently marked to identify characters aftd/or 
functions controlled. j 



4.3.2 Student and Administrative Terminals I 

A* mix of alphanumeric-only, alphanumeric/ graphics, and 
alphanumeric/graphics/color terminals shall be provided for student and 
"admini strati ve use. The numbers of terminals of the various types shall 
be sufficient* >to provide tire functional capabilities described in Section 
3.0, and shall meet. any additional requirements imposed by character- 
istics of the Air Force-designated courses to be implemented on the 
system. An external jack for controlling a device such asa slide pro- 
jector shall be provided, if course characteristics indicate that such 
a deviee is instructional ly desirable. The resolution provided by* 
graphics terminals shall be sufficient to accommodate the course 
materials to "be implemented; Color terminals shall provide at least 
eight different colors, including black and white, with color available 
for foreground letters and figures and for background. An option to 
fill colors within graphics polygon boundaries is desirable. In addition 
to the keyboards provided for student and instructor interactions, the 

system shall also support a device such- as a T ight oe&~afc. touch .panel* 

to facilitate interaction with graphics displays. To facilitate pre- , 
paration of graphics displays, the system shall" support a digitizing 
tablet or equivalent device, 

4.3.3 Management Devices > 

» .« 

Management devices in types and numbers sufficient to satisfy 
the CMI requirements, described in Section 3„0 shall be provided. As 
rtecessary to meet the instructional 'and administrative needs, these shall 
Include devices, to read forms, to print student assignments and class- 
room operations reports (such as rosters, absence reports, and regis- 
tration data), and such other devices as may be required and cost 
effective for the designated installation site. 

4.4 RELIABILITY V \ - -< 

An effective reliability program shall be established and maintained 
within the general intent of MIL- STD- 785 A. This program shall include 
failure reporting and design changes to correct pattern failures* The 
program shall be adjusted^® suit the types of procurement and equip- 
ment, and shall be directed 'toward 'maximizing the mean time between., 
failures (MTBF) for individual equipment items. System reliability shall 
be sufficient to meet the availability requirements of Section 4.6. 

4.5 MAINTAINABILITY 

A maintainability program shall be establi|hed, within the general 
intent of MIL-STD-47GV ThiX program shall incorporate a maintenance 
concept intended to minimize* the mean time to restore # (MTTR). "System 
MTTR shall be sufficient to meet the availability requirements of 
Section 4J5. \ 
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:4.6 AVAILABILITY <** \ 

*■ System ayaf lability shall be defined as follows: 

"** ■ "*.f • f ' * ' ; ' *• ■'.••*' ' s » * " 

' System Intrinsic Avaiiabilit/ =* MTBF (MTBF +*MTJR) • * 

. ' " • « .; . *. . 

where MTBF "fe defined as the system mean time between failures (i.e*, . 
the MTBF resultfng from combining the MTBFs pf all, the individual equip- 
ment, items), and MTTft 1s- defined as the mean time to restore the system, 
after a failure to" any equipment in the system. ^ •» \ • 

" r ? The Systepi Int<Hnsic Availability for the low-cost* PAI/fMI system 
shall be\at least/Q^98- y> . ~ * 
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